In this 3-part series - Engineering a New Job, I share my personal approach to job changes - not only finding a new role, but making it a success.

Previously in Part 1 and Part 2, I shared my approach on preparation and the hiring process, respectively. This last part focuses on the onboarding process - how I can quickly settle into a new role.

Ramp up

Just as the foundation determines the height of a skyscraper, a strong first 3-6 months of ramp-up in a new job can meaningfully decide the ultimate level of success in it. As a result, it is essential to make the most of this limited-time opportunity, when there tends to be the most support but the lowest expectations.

Get a head start

As part of the hiring process, especially around the offer stage, I’m already thinking about setting myself up for success, if I were to join the team.

• I would try to talk to as many people I would work closely with as possible

• I would specifically ask for material or resources to start looking into

• I would do even more research on the company/team and get a sense of how they work

So hopefully by the time I officially start a new role, I’m already somewhat familiar with the environment.

Explore then exploit

The number one priority of the ramp-up period is learning, so this is the time I have a heightened sense of anything that allows me to pick up new things more quickly, including:

• Having 1 on 1 with as many people as possible, even if I might not directly work with them straight away

• Do at least one pass on as many resources (docs, videos, training videos) as possible

• Ask as many noob questions as possible, there’s only a short window of time I can do this without getting frowned upon

• Volunteer to do as diverse a set of tasks as it makes sense to, e.g. write/modify documentation, help others debug, attend meetings, review code even if I don’t understand, shadow an on-call rotation

I prefer a Breadth First Search approach to onboarding, where I would survey the lay of the land first before I start focusing on specific areas. I feel this allows me to participate in more conversations earlier, and gives me a broader sense of priority, so that I’m not tunnel visioned on something that may turn out not very important.

As I build a more and more detailed mental model of the work, I can focus more time on specific areas.

Start small and build momentum

As I go from explore to exploit, I would also go from small problems to bigger ones, and this combination has worked really well.

Tackling small problems early on allows me to gain more cycles of “full-stack” processes, without being bogged down in one specific step. This way, I’m able to change code and quickly see how it gets deployed in production, which gives me exposure to all the tooling, from source control, to change management, and all the way to the monitoring stack.

Being able to register wins early on, however small, is a huge confidence boost for me, as well as for everyone around me. This sets off a snowball effect of me being able to tackle bigger and bigger problems, with higher and higher expectations.

Build Relationships

Having productive working relationships with coworkers are extremely important, and first impressions can very much set the tone for them. That’s why I make extra effort in starting off on the right foot with everyone on a new team:

• I try to connect with everyone on somewhat of a personal level. My approach here is to try to find commonalities with everyone, whether it’s sports, certain experiences, or opinions. If I can identify something to bond with for everyone I talk to, that’s usually a great start.

• I observe & follow the norms on the team. This can include small things such as what time people usually start and leave work, what time they lunch, what analogies they tend to use, do they small-talk, etc. It’s subtle, but once I follow the unspoken norms, people treat me as one of their own, and things get easier.

• I try to be extra agreeable. Even if I have some strong opinions, I try not to ruffle too many feathers too early on - unless I’m specifically asked about them. Even then, I leave plenty of nuances.

• I try to observe the power dynamics. Figuring out who tends to agree with whom, and whose opinions matter more in what situations would also allow me to find my way round the new team quickly.

Hold off on big ideas

Being the new person, it’s natural to come in with an abundance of enthusiasm, and frequently, this manifests in big ideas. Unfortunately, being the new person also means I have the least amount of context on the work. Most of the time, there’s something more nuanced as to why things are how they are, and why my big ideas are less practical than they first seem, I just don’t know about them yet.

As a result, I’ve learned to hold off on publicizing my big ideas, especially if they seem too obvious. Instead, I note them down for myself to look back on in 6 months. If they still apply, I will act on them then.

Summary

Adapting to a new role is not easy, and I’ve found the onboarding process to be a tricky balance between learning as much as possible, while also trying to impress everyone and establish myself. In attempting to be successful at both, I would:

• Get a head start even before I officially join

• Get as much exposure as possible when I join before digging deeper

• Start small and get early wins to build momentum

• Try to connect with everyone, fit in, and build meaningful relationships

• Document big ideas, and act on them later

In this 3-part series - Engineering a New Job, I share my personal approach to job changes - not only finding a new role, but making it a success.

In Part 1, I talked about the search & preparation. This second part focuses on the hiring process - how I gather all the information I need to make a decision. Onboarding will be covered in Part 3.

Being evaluated

The classic part of the interview is when the interviewers evaluate me for the role. There is already so much literature on how to prepare for technical as well as behavioral interviews, so I will not focus on it here.

Evaluating the employer

Instead, I’ve found it equally, if not more, important to find out as much about the company as possible, so that I can feel confident in my decision to join (or not join), if I’m in the position to do so. Here is a list of questions I try to find answers to before I sign an offer (some questions are more applicable to start-ups than others).

Overall Company

• Is this company/role within the parameters of what I’m looking for as I’ve set out in my preparation (e.g. in terms of its size, age, funding stage, industry, tech stack, location)?

• How is the company doing financially: how many rounds of funding and how much has it raised? How are the metrics looking (e.g. revenue, runway, burn rate, burn multiple)?

• What are the current plans for growth (e.g. revenue, user, product, head count)?

Product & Distribution

• What are the products/services the company is building? How big is the total addressable market? What does the competitive landscape look like?

• How do the products/services generate revenue? What does unit economics look like? How much is a typical customer’s lifetime value?

• How are customers acquired? How much does it cost to acquire a customer? Are there sales processes involved? If so, how long is the sales cycle and what is the typical/largest deal size? Do we do customizations for special customers?

Culture

• How are individual performance evaluated?

• What is one thing that is common in the industry but is not done here, and vice versa?

• How are project/strategy/personnel decisions typically made?

• What were some common reasons why someone has left the team?

Team/Role

• What’s the background of the leadership team? Have they done similar jobs in the past? How experienced and/or motivated are they?

• Who’s the manager I’ll be reporting to? When/Why did they join? What’s their style and approach to management?

• How big is the team I’ll be on? How big is it planning to be in 6 or 12 months? What’s the distribution of background & experiences of the people on this team?

• What are the most important qualities needed to be successful in this role?

• How does this team manage/organize its work (e.g. how is what gets to be worked on decided)?

Engineering & Projects

• What is the main tech stack this role will be working with?

• What are some of the main technical challenges facing the team at the moment?

• What are the first few things I would be working on if I join?

• What engineering practices is the team looking to do more of (and less of)?

• What does on-call look like?

Compensation

• What is the comp structure? Are there bonuses, stock refreshers? How are they determined?

• For stock options, I rely on this amazing piece on start-up options, so I would gather all the details on strike price, post-termination exercise window, support for early-exercise, etc.

Many of these questions can be answered fairly early on, perhaps in the first one or two interactions with the company. For the rest, there are a few avenues to find out the answers:

• Usually towards the end of each interview, there’s a customary 5 minutes to ask the interviewer questions. I would typically slip in a couple engineering related questions here if the interviewer is an engineer.

• After the final round, especially if feedback is generally positive, I usually get or will ask for a chance to talk to the hiring manager again. I tend to take this opportunity to get to know the hiring manager more, and ask more specific questions about themselves and the immediate team.

• In the offer call, I usually hash out all the details and questions about compensation, and ask for a follow-up email summary.

• After a verbal offer, companies sometimes offer additional opportunities to speak to higher level leadership and/or a broader team in the company. That’s another chance of getting a read on the vibe of the people there.

Negotiation

How to negotiate offers is also a very well-covered topic, so I’m skipping over the techniques here. However, instead of treating negotiation as having a one-dimensional goal of getting more total comp, I see it as a process to align on expectations, which is the first step in setting myself up for success if I join.

Negotiating on all fronts

If I’m going through the work of changing jobs, I’d like the new job to be a meaningful improvement in some ways or another. It doesn’t have to be money, but I would try to get as many of my needs met as possible. I would potentially ask about:

• Is there flexibility on time off, working location, working hours (e.g. for child care), or job title?

• Is there flexibility between cash v.s. stock/option v.s. bonus?

• Is there flexibility in picking specific projects/teams/areas I’m more interested in?

• Is there support for education, conferences, or other professional training opportunities?

• Are there restrictions on my other activities, e.g. teaching, or writing, or contributing to open source projects on the side?

• Is there continued legal/immigration support?

• Are there extra benefits outside of medical/dental/vision?

• Is there a referral bonus for referring candidates?

For bigger companies, some of these aspects are already written policies that are difficult to change or make exceptions just for one individual. Start-ups, on the other hand, might be more willing to accommodate them, especially if they think I’m a strong candidate.

Even if nothing ends up changing, aligning with the hiring manager on what’s important to me upfront can minimize surprises after I join, and make the onboarding process smoother.

Have multiple offers

In terms of getting a better compensation package, having multiple offers is very useful, especially for big companies. This is why it’s preferable to plan and time the interview processes so they line up around the same time.

Summary

To summarize, besides trying to impress, I try to gather as much information as possible during the hiring process, and start setting myself up for success if I join:

• Get a sense of all aspects of the role and the company by asking about its product/services, revenue, culture, the team, and its engineering practices

• Negotiation is more about setting up expectations on what’s important to me than getting more money

For Part 3, I will share my approach to quickly onboarding to a new team.

In this 3-part series - Engineering a New Job, I share my personal approach to job changes - not only finding a new role, but making it a success.

This first part focuses on preparation - what I do before the first conversation with a new potential employer. The rest of the series will focus on the hiring process (Part 2), and onboarding (Part 3), respectively.

Set Goals, Expectations & Timelines

There have been times when I was just browsing jobs, or someone reached out with a very interesting opportunity. However, I’ve had better results when my job search was planned, for a few reasons:

• I can develop a coherent search strategy that allows me to focus on the most promising companies

• The process goes more smoothly with recruiters and companies when they know I’m more committed to actually making a change

• I can line up the timelines, so potential offers arrive around the same time, which makes a big difference for comp negotiation

Therefore, even before I update my resume, I make sure to have a good sense of:

• Why I’m looking for a new opportunity in the first place

• What are the top 2-3 most important things I’m looking for in my next opportunity

• What type of companies I’m looking for - e.g. size, funding stage, industry, vibe, etc.

• What type of role I’m looking for - e.g. technology, responsibilities, type of project, etc.

• What is absolutely non-negotiable - e.g. minimum comp, location, certain policy etc.

• When I’m looking to make a decision

It feels uncomfortable to pass up seemingly great opportunities, just because they don’t quite fit my target profile. But just like dating, this is probably for the better, as it avoids spending time on something that is unlikely to work out in the end.

Resume & Preparation

As I nail down the type of role I’m looking for, I try to get myself in the best shape to fit the role.

Resume

In my opinion, the resume is one of these paradoxical things that have an asymmetric payoff. A good one can barely make you stand out, but a bad one can definitely fail you straight away. Therefore, even though recruiters or hiring managers usually don’t spend more than a minute looking at it, it’s still important to have a clean and professional one.

There is no shortage of resume templates or advice, so I won’t delve into the details here. My general approach is to have a solid baseline, with a few impressive-looking highlights here and there. That is usually sufficient.

Personal Introduction

I’ll be introducing myself many times throughout the hiring process, so I tend to have it ready early.

I prepare a longer (~5-7 minutes) introduction that is typical of the first part of a recruiter call. This covers a brief overview of my work history, why I’m looking, what’s important for me, and the highlights of one or two recent projects. This usually hits most of the points for a recruiter.

I also prepare a 1-2 minute intro that I use at the beginning of every other interview. This may have a few variations depending on the interviewer, but the main goal here is to have something memorable or relatable to the interviewer, so it's easier for them to remember me among all the candidates.

Technical Interview Prep

There’s no magic here. I read technical books & blogs for system design questions, and use Leetcode for coding questions.

Search

Personally, I’ve found it more effective to curate opportunities on a continuous basis, rather than trying to find them when I’m ready to move.

For example, I periodically check LinkedIn to get a sense of the hiring market, I try to maintain somewhat of an ongoing relationship with the recruiters who’ve reached out to me, especially the ones I’ve worked with before, and I definitely try to stay in touch with the hiring managers who seemed to have liked me in interviews. This way, I can start with already warm leads when I’m ready to interview.

In addition, I will also try to expand the opportunities through the following channels.

Referrals

Referrals are by far the best channel to enter a hiring process, especially for smaller companies. If I’m interested in a company, I always ask around to see if I can get a referral. I almost always get to talk to someone from the company when being referred.

Agency Recruiter

I’ve found agency recruiters to be very helpful for a number of reasons:

• They have access to many more opportunities that aren’t well publicized, for example, some companies do virtually all their hiring through recruiters, and don’t even look at direct applicants

• They will advocate my candidacy to the company and prep me for interviews, which can be quite helpful

• They also tend to help negotiate better offers, as many are paid based on a percentage of the first year base salary

The hard lesson I learned here is to not get pressured into taking offers that I’m not very comfortable with. Closing a candidate is a big deal for recruiters, so they tend to be somewhat pushy in the last mile. But if I insist on being rigorous in my process, it is generally respected.

Job sites

Of course, there are the typical job sites, where I can directly apply. I’ve found the following useful:

• YC’s work at a startup is great for start-ups

• LinkedIn has a very large collection of jobs

• Wellfound (formerly AngelList) has some good ones too

Research the companies

The final step before I enter myself in a company’s hiring process is to do some research on them. Typically, I’d look at:

• The industry/market they’re in

• Their funding stage and size

• The general vibe, e.g. by talking to an employee if I have connections, or read press or social media about them (e.g. TechCrunch, or Blind)

• Their engineering blog if they have one

• Their self-stated culture & values, i.e. at a high level, how do they operate, what do they care the most about

• Their engineering comp/level structure (e.g. levels.fyi)

And most importantly to me, I look at a company’s leadership team and their experience. This could be their founders for start-ups, or their org leaders (if known) for bigger companies. Here, I try to get a sense of founder/market fit and make a superficial assessment about their chance of success. Additionally, I would also contemplate potentially working with them and see if I have a chance of building a strong relationship with them.

Summary

To summarize, before I start entering hiring processes, I would:

• Make sure I have a plan for my job search in terms of target roles and timelines

• Spend time getting my “marketing” materials ready, and technical chops brushed up

• Curate a list of promising opportunities, and

• Do some research on what each of these opportunities entails

For Part 2, I will share my approach to the hiring process.

Prompted by the U.S. Supreme Court’s decision on June 29, 2023 about Harvard and University of North Carolina’s admission practices, and its immediate discussions, I’d like to contribute my own thought process, starting from first principles, to this conversation.

Axioms

Axioms are statements that are taken to be true. I hope the axioms I put forth below are uncontroversial.

Ax. 1: There is no universal definition of fairness

There doesn’t seem to be a universally accepted standard or definition of fairness. What one person considers fair might be grossly unfair to another person.

Below is a classic graphical illustration of “equality” v.s. “equity”1, that highlights this point in an extremely simplistic case. I contend that, while this hits a great point, it’s not sufficiently instructive for real world situations.

What if there are only 2 boxes instead of 3? What if the smallest kid still can’t see over the fence standing on 2 boxes? What if there are 10 kids and only 5 boxes?

I imagine different people could come up with different, albeit all “fair”, arrangements to the hypothetical situations above, which are closer to the real world.

Ax. 2: Resources are limited

Not everyone can get everything all the time. Perhaps this is tautological because what people want grows with what they have, so maybe it’s just a general truism that they’ll never be fully satisfied.

Even practically speaking, there simply isn’t enough “stuff” to go around. We don’t have the landmass to give everyone in the Bay Area an acre of land. We don’t have enough doctors to see everyone at a moment's notice whenever they need it, even with costs aside. And of course, there’s only 1 Ed Sheeran, so if you want to listen to him live, you have to squeeze into the limited space in his vicinity.

Ax. 3: There are inherent differences in people

Not everyone is exactly the same.

Some differences are biological. Some people are taller, some are lighter-skinned, some are born with a uterus, and others aren’t.

Some differences are social and cultural. With different backgrounds, families, and experiences, people speak different languages, develop different skill sets and interests, and have different preferences and opinions. 

Not everyone actually wants to own an acre of land in the Bay Area even if they can afford it, and certainly not everyone is an Ed Sheeran fan.

Propositions

If we accept the axioms to be true, we can deduce further statements from them that logically follow.

Prop. 1: Inequality seems inevitable

If we accept people being different (Ax. 3), and we respect their differences in preferences in the society, they’ll end up doing different things, leading to unequal outcomes.

On the other hand, if we force people into the same things, some people will like it more than others and derive more enjoyment out of it, in my opinion, that also makes them unequal.

Prop. 2: Generalization seems inevitable

Given that people are different (Ax. 3) and resources limited (Ax. 2), it seems impossible to have an entirely personalized societal system, where we decide every matter based entirely on individual circumstances, i.e. the “the law shouldn’t apply to me, because I’m unique” argument.

For example, it’s difficult to imagine everyone getting a personalized road test for their driver’s license, a personalized dose threshold for whether they’re driving under influence, and a personalized speed limit. 

It’s not even clear where to draw the line on the personalization spectrum.

Car insurance companies already take into account factors such as the driver’s age, marriage status, years of driving experience when calculating the premium. But one could always argue that they’re better than their peers. Should they consider even more factors? But that’s still making a decision about an individual based on a generalized behavior of a group, albeit a smaller one. Should they conduct an in-depth interview for every person asking for a quote? How about a dynamic premium pricing on a per trip basis? How far should we go?

Prop. 3: Winners/losers seem inevitable

Given the limited resources (Ax. 2), even taking into account people’s preferences (Ax. 3), there will always be competition for the best resources (by some common standard). And with that, there will always be winners and losers.

The best schools, universities, companies, and other social groups will have more people who want to join than they have capacity for. As a result, decisions will have to be made about who’s in and who’s not, and they will have to involve some level of generalization (Prop. 2) when matching the applicants with their selection criteria.

The people who lose out on these opportunities will naturally feel like they were unfairly treated, given the relative subjective nature of fairness (Ax. 1).

So what?

If we accept these axioms and follow them to their logical conclusions, the situation seems rather hopeless. Allegations of unfairly distributed opportunities for scarce resources doesn’t seem it’ll go away, and the Harvard/UNC case seems just like any other example of this nature.

So what’s the big deal?

It seems to me that the significance of this case comes mostly from the complex history of racial relations in the United States, which made the conversation more political than rational, and extremely emotionally charged. As a result, both sides are accusing the other of immorality while, in my opinion, it comes down to different definitions of fairness.

For African Americans, with the multitude of historical discriminations against them in the U.S., including slavery, Jim Crow laws, and more, giving them more of an opportunity for higher education seems a good thing, both for themselves and the racial integration and diversity of the institutions.

For Asian Americans, carrying the historical baggage of the Chinese Exclusion Act (1882), the Japanese American internment camps during WW2, among others, having also worked hard to be a qualified applicant of higher education, only to have their application denied because there are simply too many of them also doesn’t seem quite right.

In isolation, both arguments seem plausible, but unfortunately there are more qualified applicants than there is capacity, so some of them have to be turned away (Prop. 3).

In my opinion, to say people who may have benefited from affirmative action deserve their places simply because “they belonged”, as Michelle Obama’s statement does, seems disingenuous, because they may have displaced others who also belonged. On the other hand, to insist the selection process to be strictly “color blind” also seems inconsiderate of the social-economically disadvantaged groups who have had much less to start with in the first place.

Moving forward

I offer no solutions here, as I believe the problem of perceived inequality and unfairness are philosophically inevitable.

What I do hope to see though, is a public conversation with less emotionally charged accusations and more constructive ideations towards a more generally acceptable societal system.

After all, no African American alive today was born a slave, and no Chinese American alive today was a direct victim of the Chinese Exclusion Act.

We’ve got to move past history to build a better future.

Now we have a daunting task awaiting us with a completely non-negotiable deadline. We’ve got to name the baby.

With my wife and I both being Chinese living in the U.S., we have a specific set of criteria we’d like for the name.

1. We wanted a name with a spelling that’s both a Western name (or an English word), and a valid romanized Chinese name (i.e. pin-yin). For example, John is out because there are no Chinese characters that spell out to J-o-h-n. On the other hand, Zhongxiang is out too because it’s a recognizable word from Chinese and is not a Western name or English word.

2. We wanted the name to be 2 Chinese characters. I have a single character name, I didn’t quite like the rhythm of it. For example, Kai is a great name, but it’s just a single character.

3. We wanted the name to be a generally accepted “boy’s name” in both English and Chinese. For example, while it might be possible to come up with Chinese characters that spell Maya so Chinese readers could see it as a boy's name, it’s out because the English name “Maya” is generally perceived as a girl’s name. 

4. We wanted the name to have some meaning in both Chinese and English. For example, even though the name Luke satisfies the first 3 conditions, in Chinese, it sounds very much like a direct translation of the English name and doesn’t have a traditional Chinese meaning behind it.

5. We didn’t explicitly talk about this at first, but we realized we didn’t want a traditional or very typical name from another culture. For example, Yuri is a classical Eastern European or Slavic name. With my son not having any Slavic heritage, we didn’t want to have the dissonance between the person and the name.

First glance, this seemed like an impossible set of conditions. We thought so too. But we thought we’d give it a try anyway, and with 9 months of time, maybe we’d come up with something.

First 8 Months

We know the task is difficult, but we didn’t really put too much concentrated effort into it. We came up with and wrote down some names here and there when they popped into our heads, but none of them seemed entirely satisfactory.

We had names such as Luke, Lian, Timo, Jude. For quite some time, we really liked Timo and it was an acceptable back-up if we weren’t able to come up with anything else. It gave us some confidence that finding a “perfect” name is still possible.

Interestingly, there seem to be a lot more girl’s names that meet the conditions, for example, Miya, Lexi, Nina are all great choices.

3-4 Weeks to Due Date

With the due date coming up, we put more effort into coming up with more options. We found certain patterns that can give us a bunch of choices, such as Zimo, Zemo, Zeno, Chase, Channing but they also led to some rather uncommon ones, such as Size, Mice, Dice, Ate, Late, Mate, Make.

Of course, we asked ChatGPT for its help as well. Somewhat disappointingly, I couldn’t get ChatGPT to understand the first condition. It kept giving me Western names that aren’t valid Chinese pin-yin, no matter how I explained it.

At this point, we still liked Timo the most. But our hesitation was on condition 5. Since Timo is primarily of Finnish/Germanic origin, we’re not too sure about its cultural implications.

1 Week to Due Date

We weren’t able to make a lot of progress in coming up with more options, even with more frequent and intense discussions and brainstorming sessions.

Exactly 1 week before the due date, in the evening, my wife felt a strong wave of contractions. Even though the baby didn’t come out that day, it was a strong reminder of the imminent upcoming event.

We had to have a name ready ASAP!

With a renewed sense of urgency, I realized we didn’t have to keep banging our heads on the wall like this. We can automate an exhaustive search!

Attempt 1: Naive starter

The first idea I had was to simply generate all combinations of 2 Chinese characters and look through them to discover suitable names.

Even though there might be up to 20,000 different characters, the number of sounds (i.e. spellings) are far smaller (especially without counting the tones). Within each Chinese word, typically it consists of an Initial (that usually contains the opening consonants) and a Final (that supplies the vowels).

There are exceptions, and there are combinations that are not valid, but instead of listing out all actually valid spellings, I thought I’d use the combination of all Initials and all Finals to make up a word, and then list out all 2-word combinations.

Here’s the Python code in its entirety:

initials = ['b', 'p', 'm', 'f', 'd', 't', 'n', 'l', 'g', 'k', 'h', 'j', 'q', 'x', 'zh', 'ch', 'sh', 'r', 'z', 'c', 's', 'y', 'w']

finals = ['a', 'o', 'e', 'i', 'u', 'v', 'ai', 'ei', 'ui', 'ao', 'ou', 'iu', 'ie', 've', 'er', 'an', 'en', 'in', 'un', 'vn', 'ang', 'eng', 'ing', 'ong', 'iao', 'ian', 'iang']

whole_words = ['zhi', 'chi', 'shi', 'ri', 'zi', 'ci', 'si', 'yi', 'wu', 'yu', 'ye', 'yue', 'yin', 'yun', 'ying', 'yuan']

words = [i + f for i in initials for f in finals]

words += whole_words

words += ['ai', 'ou', 'er', 'an', 'ang']

names = [w1+w2 for w1 in words for w2 in words]

print(',\n'.join(names))

I ran this and was proven to be way too naive. Even after deduplication, there are more than 412k combinations. Unfortunately that’s way too many to look through manually. And worse, many of these combinations are not even valid!

Attempt 2: Automated filter

The obvious optimization is to write down the list of spellings that are actually valid for a Chinese character instead of using all Initial <> Final combinations.

That turns out relatively straightforward. I found this website that contains a table of exactly the valid spellings, and I simply typed up all 413 of them in a list, and saved it to a file (`words.txt`).

Immediately, we narrow down the valid 2-word combinations to 413*413=170,569 possible names. Unfortunately that’s still too many to look through one by one.

with open('words.txt') as w:

    words = [wo.strip() for wo in w.readlines()]

names = set([w1 + w2 for w1 in words for w2 in words])

print("total number of raw names to start with", len(names))

Now, since we started writing code anyway, we might as well start filtering the possible names in code as well.

After a quick look around, the first filter I added is a nice spell checker called Enchant that simply narrows the list down to valid english words.

d = enchant.Dict("en_US")

english_words = [w for w in names if d.check(w)]

print("total number of names that are also words", len(english_words))

This filtered the list all the way down to just 681 names. That’s perfect!

As I looked through this list, I quickly realized that we’re missing something. This is now a list of valid 2-word Chinese combinations that are valid English words, not names!

We don’t want to name the baby Like, or Wanna, or Women, or Rerun!

Attempt 3: Get the names in

Now the task is clear. We have to filter for English names in the valid Chinese spellings. To do that, I just need to find a list of boy’s names.

The first library I found, names-dataset, looked great. It’s a dataset of popular names in a large number of countries. It contains popularity data, and it has a nice querying interface. I made a few attempts using it to filter, but it didn’t quite work out how I expected.

When I used all boy’s names in the U.S., I ended up with still a very large number of names remaining, including all the very Chinese ones. It turns out, this data was based on Facebook, and of course, there are many Chinese people based in the U.S., so that didn’t help much.

On the other hand, when I used only top boy’s names in the U.S. to filter, even up to top 500,000, it couldn’t find a single overlap with the list of Chinese spellings.

I didn’t look much further, but I guess this is because all the names that are also valid Chinese are relatively far down on the popularity list, which is quite reasonable given the scale of the dataset.

On top of this, this dataset is rather large (2.3 GB) and takes a while to load, so I moved on.

Looking around more, I found another couple of datasets on names:

• This dataset contains a list of popular names in the U.S. by year from 2000 to 2021, based on social security data

• This dataset from a Kaggle competition also contains similar data but dating back all the way to 1880.

So I integrated them into the algorithm as well:

names_by_year = set()
for year in range(2000, 2022):
    boy_file = f'popular-baby-names/{year}/boy_names_{year}.json'

    with open(boy_file) as bf:
        names_by_year |= set([n.lower() for n in json.load(bf)['names']])

print("total number of names by year dataset", len(names & names_by_year))


names_from_file = set()
baby_names_file = 'babyNamesUSYOB-full.csv'
with open(baby_names_file) as bnf:
    reader = csv.reader(bnf)
    rows = [r for r in reader][1:]

    for r in rows:
        if r[2] == 'M' and int(r[0]) > 1940 and int(r[3]) > 9:
            names_from_file.add(r[1].lower())

print("total number of names from file", len(names & names_from_file))

all_names = names & (names_by_year | names_from_file)

with open('results.txt', 'w') as nf:
    nf.writelines([name+'\n' for name in all_names])
    nf.writelines(['--------------------\n'])
    nf.writelines([name+'\n' for name in english_words])

Conclusion

With this final version, we started out with 167,626 raw “names” that are all valid Chinese 2-word combinations (down from 170,569 after deduplication). 681 of them are also valid English words. Independently, 96 of them overlapped with popular names from 2000-2021. Furthermore, 934 of them overlapped with popular names (at least 10 boys had the name in a year) going all the way back from 1940.

After deduplication, we now have exactly 1617 names to pick from.

Here are a few that made our final shortlist: Daren, Boyan, Randi, Sinan, Yale, along with Timo and Channing from earlier.

This is now looking much better.

3 Days to Due Date

Funny how life works.

We looked at all the names on the shortlist, and we pronounced them over and over. All of a sudden, a completely new name that’s not on any of these lists came to us, and we loved it. We looked it up, and it has a great meaning. It’s easy to pronounce, and it’s cultured in both Chinese and Western contexts.

Just like that, we got a name for the baby.

However, I have no doubt that we wouldn’t have come up with it, had it not been inspired by all the work that went into it.

I suppose the takeaway from this is, hard work pays off, even though sometimes in unexpected ways.

PM: What do you mean? What’s the problem?

Engineer: On the page here (point to the page), it says “average”, but it doesn’t say what items are included in this averaging calculation. In this context, it could either be the average of set A, or set B, or set C - they are all valid possibilities.

PM: I’d assume it’s the average of set A.

Engineer: But.. It might not be clear to customers?

PM: We ran a few validation sessions with customers as well as internal teams who are experts in this part of the system. They all said they understood and didn’t have any questions on how the average is calculated.

Engineer: The customers are not thinking about this logic deeply enough.

PM: Are you calling customers dumb? It’s a very dangerous thing to call customers dumb, because we’ve built many things we imagined are important but weren’t. We need to actually listen to what customers are saying.

Engineer: I actually agree with you in general. In this case though, I’m not saying customers are dumb.

I’m saying at this part of their workflow, when they are going through this page, they’re not focused on this specific logic, so they may genuinely think they understand.

But later on when they dig into the result of their set-up here, when they realize it doesn’t match their expectations, they wouldn’t have a way to find out exactly how the average is calculated. At that point, their only option is to ask our support team, which causes more work for us.

PM: How do you know this is actually going to be a problem for customers? Just because you see it as an engineer, doesn’t mean it’ll be a problem for customers.

Engineer: That’s true, but we have historical data and questions exactly like this have come up time and again, that took our engineering team a lot of time away from their planned work.

PM: But this problem didn’t come up during user testing and validation sessions. Are you saying we can’t trust these sessions?

Engineer: When you say this problem didn’t come up, did we ask the customers how they thought the average was calculated, and they all explicitly said it was for set A?

PM: Hmmm…

Engineer: Right. So.. I’m not saying we can’t trust customer validation sessions. They have great value. But it doesn’t make sense to take what customers say as gospel, and ignore problems just because it didn’t come up in the sessions.

We are the team building it, we’ve thought about the problem for longer and more thoroughly. When customers are taking a small part of their busy day to give us feedback on a solution, we can’t expect them to uncover every problem with it, let alone the problems that are not immediately in front of them.

Heck, half the time, the customer isn’t necessarily giving it their full attention, because in that moment, what customers want, other than faster horses, is to get back to their actual work.

Unfortunately, talking to customers isn’t a sufficient substitute for thinking deeply about what we do.

In specific terms, one of the common manifestations of this principle is to have the business logic code depend on a data access “interface”, which may have different concrete implementations that the business logic code does not need to care about. This is also a key insight of Clean Architecture, inlining an illustration from the said post here.

However, anyone who worked on large systems may feel some unease about this principle, because even if we’re not sure exactly why, we have a hunch that ultimately, implementation details do matter, and not just because of Hyrum’s Law.

Latency

Latency is arguably the most obvious aspect of a function’s behavior that is not captured in its interface.

If your code relies on user_repository.get_all_users() it does not tell you how long it might this function call to return, because the theory says, it’s “a concretion” not an “abstraction”, hence not something you should depend on.

Worse, different implementations of the method may even take wildly different durations of time to return. in_memory_user_repository.get_all_users() could take a few milliseconds, whilst data_warehouse_user_repository.get_all_users() may take 30 seconds. It’s difficult to argue that the consumer of this interface must be agnostic to this, and other meaningful, yet hidden differences, in behavior.

Yes, there are techniques such as timeouts, circuit breaking, and deadlines you can use to make the system work more robust. But that’s exactly the point, that latency, and performance in general, matters.

Resource Consumption

Have you ever gotten your process/container/application killed due to Out-of-Memory? Well, wouldn’t it be nice if functions can natively be constrained to not use more than a certain amount of memory, as part of its defined interface?

And of course, memory is not the only precious resource, other factors such as external network calls, data transfer volume, disk space usage, are all factors that the consumer of an interface may want to put constraints on to align with its downstream processing.

If your code runs on a platform (e.g. Salesforce) that has hard limits, or expensive quotas, you may even want to tightly control the distribution of the quota to the internal functions, just so you don’t accidentally get a $1 million bill because one of the engineers forgot to batch up a query to the database in their quick hotfix.

Failures

And of course, running out of resources is not the only form of failure. As a consumer of an interface, you may also care about other ways a function may fail.

To this end, Java has the concept of Checked Exceptions as part of function signatures, that forces its consumer to deal with it. Even though this might be a controversial subject, the objection seems to be more about the way it's enforced in the compiler than the concept itself.

Overall, I do think the direction of supporting functions to natively document its failure modes, for example in the form of exceptions, is sound.

What now?

Dependency inversion is a very useful pattern, however given how much more sophisticated software has evolved to be, function interface definitions that haven't changed much for 50 years are starting to look insufficient.

If we maintain that the consumer of an API should truly only care about its abstraction and not concretions, the only way out is to expand what counts as an abstraction.

To that end, if you're using Python, you're in luck. liang is an open source python package that allows you to annotate exactly how long any function can take, how to measure it, and what to do with it if it takes too long.

To build more robust software, we've got to stop pretending we don't care about "implementation details".

This is a universal debate that happens everywhere, all the time, and will likely never stop. Federal v.s. State regulation, government enterprise v.s. free market, school provided lunch v.s. self packed lunch, git v.s. svn, monolith v.s. microservices, and many many more are all manifestations of the same debate. Even decisions as small as whether to refactor duplicate code into the same function, is very much a Centralized v.s. Distributed problem.

Most people can agree that there isn’t a universal way to settle the debate, so ironically, the solution to the Centralized v.s. Distributed problem will have to take a distributed form.

There are however, heuristics to follow, for example explore-then-exploit, or let 1000 flowers bloom, as well as factors to consider, when picking a side for a specific situation. Having a good understanding of these factors for the given problem can increase the likelihood of making a better decision.

Some relatively non-controversial factors to consider are summarized below.

When it comes to managing an engineering org and its work, there are specific decisions that need to be made about whether, and to what degree, each item should be centralized or distributed. Here are my opinions.

Strictly Centralized

Programming Language

For most companies, picking 1-2 languages per platform seems sufficient. Given the rather large amount of work and expertise required for maintaining a language stack, the long term implications of introducing another language, I believe very few people should have the power to make this decision, and they should take it rather carefully.

Cloud Platform

Similar to a programming language, cloud platforms nowadays not only provide great convenience in deploying software, it also affects how systems are built and communicated with each other. With a tremendous amount of expertise required and very large switching cost, this should again be an uncontroversial, centralized item.

Security Standards

In the current day and age where a data breach could realistically kill a company, it is, again, likely too much to ask individual teams of generalist software engineers to look after their own security exposure, especially if the teams are small.

Centralize the baseline, allow for small deviations

Deployment Pipeline

On a per platform basis (e.g. frontend, backend, mobile app), most of a company’s software is likely deployed in a very similar way. As a result, there’s significant leverage to be gained by building shared infrastructure to make the common path as trivial as possible.

On the other hand, there is a meaningful variance between teams that have reasons to iterate at different paces, which should be reflected in their deployment tooling. Machine Learning teams building models, or teams subject to stricter requirements such as PCI or HIPAA are classic examples requiring tweaked deployment models.

Project Management

Most product engineering teams can likely get their jobs done roughly equally well regardless of where their tasks reside. Take any of jira or Shortcut or Monday.com, it’d probably work. Occasionally, you come across one team that loves Notion so much that they copy everything into Notion anyway. Is it worthwhile forcing them out of their favorite tool? I’d say let them try and see how it goes.

Cloud Resources

Once all the teams are on the same cloud provider, I think we can give teams a little more freedom to explore which offering suits their needs the best. There’s obvious efficiency gains, and cost control benefits in having a standard process of provisioning commonly used resources (e.g. AWS S3 bucket). But on the other hand, should a team wait weeks for approval if they’d like to try out AWS Kendra (I didn’t know what it is 5 minutes earlier either) for a quick prototype? I would say no.

Hiring and Personnel Management

This one seems to have a fairly large variation, ranging from Google’s completely team blind hiring process, to the possibility of each team devising their completely independent hiring process.

Having done this for a couple of years and tried a few different options, I’m of the opinion that there should be a common baseline process and standard held across the teams, including well defined engineering levels. On top of it, if certain teams have specific requirements about their role, technically or culturally, I think the hiring manager can make the appropriate adjustments, given they have the most skin in the game on hiring decisions.

Centralize the tooling, distribute the usage

Operational Responsibilities

It seems to be widely accepted now that completely separate Development and Operations teams that don’t have much context about what each other is doing, doesn’t work very well. As a result, the team that wrote the code is typically also taking on the operational responsibilities of their system, including monitoring, being on-call, and incident management. To me, that’s a good thing.

Of course, if every team has a different way of getting logs from their system, a different monitoring solution to look at metrics, and a different way to deploy a roll-back, maybe that’s too much.

Code Repository

As long as the code is on Github, or Gitlab, or whichever hosting solution we pick, leave the teams to configure their own repos to their likings. If a team prefers merging to rebasing, what harm does it have if they only do it in the repository they own?

Testing Infrastructure

Similar to the Operations team, the QA team also seems to be on a decline, and in my opinion, for a good reason. The main argument I heard against having a QA team is that it spoils the development team into the bad habit of not thoroughly testing their code. While somewhat valid, this is actually not my biggest reason for objection.

Personally, I believe that to write good tests, at whichever level of abstraction, requires a significant amount of work to deeply understand the product and the system. Having spent the time and effort to train a group of people to be such experts of the product and system, but not have them contribute actual production code, seems to be a great form of waste.

As a result, writing tests should be squarely the responsibilities of the individual teams.

Boilerplate Utilities

There are a number of commonly used utility toolings that are fairly typical to be shared among engineering teams. From internal libraries for authentication, to distributed tracing tools such as Jaeger, to error reporting tools such as Sentry, these are common utilities that can be rather effectively templatized. Ask the teams to use the template for an efficient set-up, then set them loose.

Distributed, within reason

In my opinion, almost everything else can be distributed to the individual teams to make their own best decisions, within reason. Ultimately, there are technological as well as business constraints that teams have to respect.

This category includes but is not limited to:

• Development process, including framework, rituals, iteration cycles etc

• System performance metrics, SLOs, SLAs

• Sub-team organization and personnel

• Almost everything within the code repository, including coding style, build tool, etc

• Software frameworks (e.g. fast-api), communication interfaces (e.g. grpc)

Conclusion

Coming from a fast-paced start-up background, my default is always to allow for distributed decision making until it proves too much. There are a few reasons for this.

First, the distributed model optimizes for pace and learning. When something is done locally, it tends to get done faster, and can iterate more frequently. The ability to trial and error quickly is the one main advantage smaller companies have against the more established competition. This is exactly the Innovator’s Dilemma.

Second, distributed decisions do not automatically lead to inconsistency. When something is tried on a smaller scale and finds success, it can spread rather quickly. A grassroots solution can easily trump a centrally designed solution forced down to the teams’ throat.

However, as the size of the company and engineering team grows, there will be a need to centralize more items, as consistency and efficiency gains more leverage, and as higher level engineers chase more impact, which at some point, can only be achieved via doing centralized work.

As 2023 rolls around, here are 8 concrete actions an engineer can take to boost the chances of a promotion in a start-up environment. Hint: do more with less.

Deliver with consistent output

First thing first, in order to get a promotion, it’s fundamental to demonstrate mastery at your current job. Delivering regular and consistent work products, whether it’s in the form of code changes, documentation, or prototypes, can quickly build confidence in the team that you’re someone that they can count on. Personally, I set a goal to complete at least 1 piece of work that I could demo to the team everyday.

With the reputation of being a consistent and reliable engineer, you’re much more likely to be given bigger and better opportunities, as you’ll be deemed more likely to succeed in them.

Tackle a “oh-that-one” problem

Now instead of waiting to be given an opportunity, what’s even better is to go out and grab the opportunity yourself. This is especially valuable in a start-up environment.

Every team has a few of these nagging problems that, perhaps due to its complexity, or messiness, or high risk, everyone knows it should be done, but nobody is actually doing it. 

That’s your opportunity.

Nothing gets more publicity and gains more respect than being the hero or heroine that slays the proverbial dragon.

Send progress reports to everyone who might want to know

But the problem is, nobody would know the dragons you’ve slain if they don’t hear about it.

Whether you’re working on a long-running project, or just a quick fix, always keep everyone in the loop on what’s happening. It’s an interesting paradox that the more you let the stakeholders know what you’re doing, the more you’ll actually be given the autonomy to make decisions.

Concretely, a daily blurb to your project lead, a weekly summary to your manager, and a quarterly update on the projects you worked on to the team will get you further than you think.

Proactively help a teammate

In addition to doing a good job and letting everyone know about it, a good engineer also makes other people around them better.

Starting small, simply helping another teammate without being asked to is a great demonstration of leadership.

Everyone gets stuck at some point, maybe it’s an incompatible dependency, maybe it’s an authentication issue with a new tool, or maybe it’s a failing test.

If you hear about them, reach out to the person to help, before your manager asks you to.  You will not only build a stronger connection with the person you help, but will start developing a reputation of being helpful. The snowball will build and you will derive great impact from helping others.

Take something off of your manager’s plate

Other engineers are not the only group of people you can help. Your manager will appreciate it too.

In bigger companies with a formal promotion process, your manager is likely the person who builds a promotion case for you. In a start-up, your direct manager would have even more of an influence on that decision.

What can be more effective in giving your manager more reasons to help you than allowing them to do less work?

Ask your manager what they can offload to you in a 1-1. Ask why not if they’re hesitant. At the very least, you can get candid feedback on what you need to do to be given more responsibilities. 

Advocate for a product change

Ultimately, the value engineers provide can only get through to the customers in the form of the end product. As a result, experienced engineers are expected to have an impact not only through code, but directly on the product as well. It’s like race car drivers are expected to provide valuable feedback on how to build a more performant car.

As you’re working on every feature, take a minute to stop and think if there’s anything else customers might want to see as part of the interaction. Even simply asking a clarifying question about an edge case is a great step towards building a strong product sense.

More often than not, a strong product sense is a box you need to check off to get to a high level engineer.

Demonstrate your understanding of the technical vision

In order to get a promotion, you almost always need to demonstrate you can already perform at the next level, because promotion is as much of a recognition of your past work as a vote of confidence in your future work.

And the future is about a vision. Managers are unlikely to give an engineer more influence and scope of work if they’re not confident the engineer is in agreement with what the future of the team holds in their mind.

As a result, demonstrating your “loyalty” by actually taking a step towards the technical vision will be a big plus in your manager or tech lead’s eyes. It could be as small as refactoring a couple classes to conform to a new coding standard, or as big as creating the skeleton of a new API service.

Anything that gets the system one step closer to where it needs to be is a win for you.

Ask for feedback

Ask, and you shall receive. Getting continued feedback, positive or constructive, is the fastest and most effective way of growth.

Ask your peers what you can do more of, ask your tech lead where you need to improve, ask your manager how you can be more effective. Ask everyone else you work with how you can make their lives easier too.

Ultimately promotion is just a proxy, what’s actually important is growth. If you never ask, you’ll never know.

Wen worked hard, consistently hit his goals at work, and gained a great deal of technical competency in the first 18 months. A timely promotion to Software Engineer II came just in time for a confidence booster.

Another year in, after settling into a steady rhythm of some meaningful coding and a fair amount of maintenance work, Wen is starting to get bored and is ready to make a bigger impact elsewhere. Perhaps a start-up? Surely with 2.5 years at a well respected company and seeing some highly sophisticated systems at work on a large scale, he’d be able to make a difference at an earlier stage start-up.

Many companies think so too. After comparing a few offers and a round of friendly negotiation, Wen is now excited to join a series A start-up as a Senior Software Engineer. He had to take a pay cut on the base, but with the Senior in the title and the equity options, Wen has convinced himself that he’s made the right decision.

However, the honeymoon period didn’t last for long. Just a couple weeks in, Wen is starting to get really worried.

To start with, there are so many problems that people don’t even seem to see! The code is pretty messy, none of the Javascript is typed, tickets have almost no description, deployment takes hours, and where’s the documentation anyway? Don’t even get me started on the big monolith, this is almost 2022!

And these are not even the worst part. What really got to Wen was when he put together his thoughts into an architecture proposal and shared with his manager. “We appreciate your effort”, said the manager, but somehow, someway, all of the suggestions were deemed as either “irrelevant” or “too much effort, we don’t have time for this”.

Wen is upset, angry, and a little confused. “I’ve seen how these things should work, I’m trying to help, why wouldn’t they listen??”.

This is what I call the Intermediate Engineer Syndrome. And Wen is not alone.

See this doesn’t happen to entry level engineers because they don’t have much expectations on how things are supposed to work, and this doesn’t happen to true senior engineers because they’ve seen a variety of ways of doing things and can empathize with wherever the company is at in their life cycle.

The danger is with somewhat experienced (i.e. intermediate) engineers joining a new company. Onboarding them smoothly can truly be a challenge, and a massively important one.

The first 3 months of a new Intermediate Engineer’s time at a new company is a special period of time that requires extra tender loving care. This is the time that they have the freshest ideas and are most enthusiastic to make a difference and establish themselves, but this is also the time that they have the least amount of context on why everything is the way they are. Without enough encouragement, they’ll feel not heard and not valued just like Wen. Without enough guidance, they can really rock the boat and drag the team, as well as themselves into a big hole and hurt the team’s productivity.

What should we do?

Set clear expectations early on

During the initial onboarding process, or even as early as during the hiring process, be earnest about where the team is in terms of processes, tooling, and everything else, so when new members join, they can already have a good sense of what strengths and weaknesses the team has.

Share context as much as possible

Ideally, there is a trail of record on when certain important decisions were made, and why. Lack of that, try to share as much context with the new joiner as possible, either in a verbal or written form, to help them understand the rationale behind prior decisions. Every team has a historian. Find them, and let them tell the tales.

Encourage ideas, but don’t guarantee immediate action

Related to setting clear expectations, we need to vigorously encourage new ideas from new joiners, but be clear about the fact they might not all be implemented right away. Talk through them with the person if time allows. Ask them to elaborate. This shows curiosity and respect. And sometimes, they’d reach the same conclusions as what’s already there!

Write them down, and play back later

What I found has worked wonders, is to simply write all these new ideas down, without filtering. In a few months time, play them back to the person and ask them if they still think they should be implemented. With a few months of more context, chances are that they can already see that many of these ideas are not necessarily the best fit. Letting them say no to themselves, and potentially inspire them to come up with new, better ideas.

What this describes is how I, Yi Zhang, prefer an engineering organization to be structured. There are certainly trade-offs, but this is what I believe to be a good balance in optimizing for the areas I care more about, without overly sacrificing others.

Of course, these opinions and preferences are heavily biased by my personal experience, education, readings, and mentorship I received over the years.

High Level Goals

The end results that we’re trying to achieve with an organizational structure are:

• Alignment: Make it easy to communicate what Success means for everyone

• Speed: Help everyone to move towards Success as fast as possible

• Risk: Minimize the risk of failure to achieve Success

Principles

These are general principles that should help guide how organizations are structured:

• Autonomy: Encourage people closest to the work to make as many decisions as possible. They are most familiar with what’s involved. They have the most local context and information. Let them be creative and find the best ways to do their jobs.

• Self-Sufficiency: On the other hand, make sure people doing the work have easy access to everything they need. Notably, domain/technical expertise, stakeholder, tooling, higher level decision making rationales, etc., should all be easily available.

Organizational Structure

With these goals and principles in mind, here are the building blocks of how I prefer what the organizational structure looks like.

Team

A Team is a permanent group of people (cross-functional) who share a common customer-impacting goal. This could be in the form of a Mission Statement for the long term, and in the form of OKRs for the short term.

The mandate of a Team is to do everything it can to accomplish the goal.

Teams are owners of resources (e.g. code, engineers, cloud infra), and are the context within which all work is prioritized and carried out.

Every engineer has to belong to a single Team, and is primarily dedicated to achieving the Team’s goals. Therefore everything they do should be reasonably justifiable in terms of the Team’s goal. This helps Alignment.

Each Team sets their own goals and priorities, with discussion open and feedback taken from anyone interested to contribute, especially their stakeholders. This promotes Autonomy.

Teams can be cross-functional, including members from other functional groups (e.g. product manager, customer success, etc.), but each Team has to include a lead, who serves as the ultimate responsible party for the Success of the Team. Typically, these would be “Engineering Managers”.

Guild

A Guild is a semi-permanent group of people (typically only engineers) who share a common interest across multiple Teams.

The primary function of a Guild is to share knowledge across Teams. Typically, engineers who work on similar technologies can form a Guild (e.g. mobile engineers, DBAs, SREs, or Testing). Guild membership is voluntary but generally beneficial for relevant engineers. This systematic spread of knowledge helps manage Risk.

Guilds should hold regular meetings to gather, synthesize, and disseminate Guild specific knowledge. Guild meetings should also be the arena for organizing cross-team initiatives, including goal-setting, decision-making, progress report, coordination and so on.

However individual Guild members should advocate, prioritize, and carry out this work in their own Teams’ context. This preserves the Teams’ Autonomy while promoting Self-Sufficiency by connecting some member(s) of the Team to their peers.

Each Guild should have a lead, who serves as the coordinator of these knowledge sharing, and when necessary, the driver of these cross-team initiatives. This is generally a safe but challenging ground for testing the leadership qualities of Senior Engineers looking to move into a Staff Engineer or Engineering Manager role.

Guilds are not advised to own long-living resources (such as code). The primary beneficiary Team of such resources should typically be the maintainer. Though occasional exceptions can be made with some combination of permanent Guild, very light-weight code, and/or strong Guild leadership.

Task Force

A Task Force is a temporary group of people (cross-functional) who are set out to accomplish a very specific task in a pre-defined time frame.

The necessity of the Task Force arises from non-trivial, cross-team pieces of work that cannot be accomplished within the scopes of a single Team, but demand greater dedicated effort than feasible for a Guild. Typical examples can include significant testing infra in a shared code base, a light-weight internal framework, or a data pipeline prototype.

Task Forces should be formed thoughtfully with the timeline and members being very good matches for the specific tasks. Ideally all members carry out work exclusively for the Task Force on a full time basis for the duration of their membership. This facilitates the Speed of execution of these tasks, and allows the Task Force’s Self-Sufficiency.

A Task Force should be converted to a Team if the scope of work proves to be long-lasting and customer-impacting. Otherwise it should disband upon the completion of the tasks.

Members would typically resume prior positions in their respective Teams after the Task Force’s dissolution. And long-living resources (such as code) can be distributed across Teams or are passed to the primary beneficiary Team. 

Tribes (Optional)

A Tribe is simply a collection of Teams grouped by the proximity of their ownership area.

As organizations get large, it becomes difficult to keep track of all the Teams on an individual basis. Furthermore, Teams may naturally form cliques in terms of the frequency and bandwidth of their communications.

It might be worthwhile to formally recognize the proximity of certain Teams compared to others to help further with Alignment on their shared goals. This also promotes sound Software Architecture per Conway’s Law (see below).

Tribes may have leads to facilitate communication, goal alignment, as well as possibly driving engineering initiatives at the Tribe level. They are commonly given the title “Director of Engineering”.

Conclusion

Always remember Conway’s Law:

Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.

As soon as people are in different groups, their level of communication will be reduced, and the systems they work on will diverge.

Always have everyone building the same (sub)system stay in the same room.

FAQ

Q: What’s the whole point of this?

A: I really really hate blocking other teams and getting blocked by other teams. The number 1 thing this structure optimizes for is allowing Teams to be as independent as absolutely possible, so they can move as fast and make as best decisions as possible.

Q: What’s the expected size of a Team?

A: Conventional wisdom dictates 5-10 people. General principle is that as soon as there are too many people to have shared goals that everyone on the team can truly meaningfully contribute to, it should split.

Q: What happens when a team is formed without a sufficient mix of skills?

A: What else can we do other than hiring people with the right skills to the team? Short term perhaps you can borrow or share these people with other teams, but I can’t imagine that being a pleasant solution for anyone involved.

Q: What’s wrong with a centralized SRE team that behaves as internal consultancy to help product teams?

A: Two things. One, the central SRE team is further removed from the customers, if there’s no customer representative on that team, they’re less likely to empathize with customers and prioritize work that’s more customer impactful, and I really really believe we should put customers as high priority. And two, internal consultancy has some of the same drawbacks as external consultancy. From the product team’s point of view, with the internal consultant SRE not as intimately tied to their goal, and not having the same level of context, they can’t do as good of a job.

Q: Who would build these deployment tooling, log aggregation, monitoring, and all the shared utility things that everyone will need?

A: Great question. First of all, I challenge that everyone will want/need the same thing, and I challenge that a central SRE team would build these better than people within these teams can. Practically this can be developed by a Task Force and maintained by the SRE Guild. If a Team has capacity, they would be welcome to build and maintain it too.

Q: So EVERY SINGLE team has to be customer facing?

A: That’s the idea that I’d like to take as far as conceivably possible. I concede that at some point when the company is big enough, we might want/need an internal tooling Team. And I want it to be because at that point, some people have volunteered to have built some great tooling that we’d like them to maintain rather than any other reason.

Q: What if none of the Guild members can convince their Team members to prioritize the initiative of their Guild?

A: The idea is that if they are important enough and they’ll benefit the Team, the Team should recognize that and would be willing to spend time on it. But only as much as their own context allows though.

Q: What if there’s something really important that all teams really need to do, like for Compliance and InfoSec?

A: Establish a Guild that drives it, and let the Teams know that this is important. If they need to be mandated to occupy the top of the backlog, so be it.

Q: What if a Team needs some help on SRE, but not full time. Should I hire an SRE for the team?

A: If you can make do without one and accomplish your goals, then great. There’s also nothing stopping you from trying to consult with other SREs on other Teams, just know you’re not their top priority. And honestly, if you hire a great person, they’ll find enough work for themselves.

Q: What if the only SRE on my team goes on vacation?

A: I think it’s pretty reasonable to ask someone else on the Team to learn the basics to cover for them on a temporary basis.

Q: This structure seems too extreme, too rigid, are there exceptions?

A: Yes there are. There are certain functions that seem to make more sense to be centralized. Security might be one, because it has to be more standardized across the board, and there’s definitely not enough work for every Team to have a security person. On top of that, it’s a domain that’s more specialized and harder to train a generalist engineer to do. But I struggle to think of any other ones.

References:

TransferWise

Spotify

BaseCamp

Valve

The Five Dysfunctions of a Team

John Cutler