Velocity is one of the most common metrics in software development, but very often is misused. Here's what the process managers get wrong and how to use velocity to the benefit of the whole organization.

What does velocity measure

One of the most common practices in software development is estimating tasks (or so called "user stories" – terrible name, won't use it here). There are many different ways to do it, but the most common way I've seen is:

• have the whole team discuss the task at hand and agree on estimation (following some subset of  Planning Poker rules)
• use a certain set of numbers to represent estimation values. The most common set I've seen is the first few numbers in Fibonacci sequence: 0, 1, 2, 3, 5, 8, 13... up to some defined maximum. An alternative to numbers is using t-shirt sizes like S, M, L etc. and though I believe it's better, it's way less common, so I'll stick to numbers in this article.

As an outcome of the planning process we end up with key-value pairs, where every task has assigned a certain number called "story points" or just "points". This numbers is meant to represent... something. Let me get back to this "something" in a moment. For now let's assume it's a positive value that the team gains once the task is complete.

Following the planning, the team works in fixed-length cycles, let's say 1-month long. At the end of every month they look at the tasks that were delivered, they add up the number of the points of each task, and end up with a single number. After a few cycles, the team calculates the average of this number in the last few months. Let's say the team delivered 90 points in January, 70 in February, and 85 in March. The average of these 3 numbers is 82, and that's the "velocity" of the team.

This number represents how "fast" the team is going. It's a single number that represents all the team's work over a few months, and this number can be used to measure whether the team speeds up or slows down. It can even be used to compare performance of individual developers, to see who's driving the team's performance and who's lagging behind. Magic ✨ right?

Well, not really. Velocity can provide value, but the way it's often used by Agile aficionados and inexperienced managers ruins the value of this metric. Let me explain how process managers fuck it up and how to actually use velocity effectively.

How process managers misuse velocity

There are many pitfalls of using velocity. In this article I'll focus on the following 4 ways in which process managers misuse it:

1. They have poor definition of story points, and often use points to indirectly represent time
2. They believe that "more = better" and they push teams to increase velocity
3. They use it as the only metric of team performance
4. They use it to measure performance of individual team members

Pitfall 1: poor definitions

Let's start with the basic – what does a single point mean? What are we actually estimating when giving tasks certain value?

My time is not your time

One of the most terrifying, and yet common, ways to estimate is to use time. Time to finish the task. Developers are asked to predict how much time a task will take them. So 1 point might be a proxy for half a day, or a day. Then 2 points are 2 days, 5 points are a week.

In one of my old videos I explain why this makes no sense. You can watch it below or keep reading for a summary:

There are multiple things that are wrong with this approach. The main is that how much time I need to complete a task is different than how much time you need. So when we choose a single value, whom should we listen to? A senior developer who's built the product from scratch? A mid-level developer who's still learning the tech stack? Or a junior that joined 6 months ago? When asked to estimate we need to have the same unit of measurement, and when we're asked how much time it'll take to complete the task we ask ourselves "how much it will take me to complete it", hence every team member asks a different question.

Some teams try to workaround this by assigning the tasks before estimating, and during estimation they ask "how much time it will take this specific developer to complete the task?" This approach is so bad it deserves its own article, but just to mention its biggest sin – it shifts from team-oriented to individual-oriented work, where everyone only cares about the tasks they have assigned to their name, instead of caring about all the tasks the team committed to.

What to estimate instead?

The only way to make estimating work is to have a common definition of what we estimate and what each value means.

For example, what I usually ask my teams to estimate is a mix of complexity and uncertainty. But because we don't have a clear definition and a way to measure complexity, we need to come up with something on our own. This is by design. Story points are abstract, imprecise and inaccurate units of measurement. We know that they don't refer to anything tangible, we know we can't use them to measure down to the tinies detail, and we know they're error-prone. That's alright, as long as we are trying to estimate the same thing, over time we'll make our estimations better.

Hence in order to start estimating tasks, we need to meet 2 conditions:

• everyone needs to consider the same thing – if we ask a group of people "how much time it'll take you to drink this bottle of water?" everyone will think about their own capability, but if we ask "how big is this bottle of water?" everyone will think about the same thing
• everyone needs to understand the unit of measurement – since we don't have equivalent of liters or kilometers in measuring task complexity, we need to give people some guardrails so that they know how to use the units they're given (we can use techniques like reference tasks or comparison)

Pitfall 2: more = better

The 2nd way in which process managers screw up the concept of velocity is that they believe that the velocity should grow over time. I don't really blame people here, in the end velocity = speed + direction, and the faster we move the better, no? Wrong! Expecting developers to increase velocity over time contradicts the whole idea of estimating the complexity of tasks.

The base of this wrong assumption is that as the team gets better, they will deliver more value over time. And that's true, the team will build similar things faster, hence they will achieve more in the same span of time. But at the same time, as the team gets better and builds more sophisticated software, they will give lower estimations to similar tasks.

Let me give you an example. We're building an e-commerce solution that up until now supported just 1 payment provider, let's say Stripe. Now our customers want an alternative, so we add an option to pay with PayPal. Currently everything is hardcoded to use Stripe flow, hence adding new payment provider requires a lot of changes. We estimate this task to be worth 13 points. Fast forward a few months, a similar request comes – this time we need to add support to Alipay. But since we had to switch from 1 to 2 payment methods last time, we replaced hardcoded values with nice interfaces and polymorphism. Hence we estimate this task to be worth only 5 points. A very similar task has become simpler for us, therefore we assign it less value.

This is why the velocity of the team over time should reach plateau and stay roughly the same, not grow (unless the team gets more people). If the process manager sees that the velocity is consistently growing despite no changes in the team, this should raise suspicion.

Pitfall 3: using velocity as the only metric

The third mistake is relying purely on velocity when measuring team's performance. As the famous Goodhart's law kind of says:

When a measure becomes a target, it ceases to be a good measure

Hence, when a manager tells developers they need to improve their velocity, they don't incentivize the team to work harder or smarter, they just incentivize the team to... increase velocity. What that means is that the team will start inflating numbers, giving larger and larger values to the same type of tasks over time, just to show that the line on the velocity chart always goes up.

There's more, though. If the only measurement of success is velocity, then nothing else matters. Quality? Who cares, approve that Pull Request and let's go! Bug fixes? Only if a bug is assigned a number of points, otherwise it goes to the bottom of the backlog. The team works long hours and can't keep it going for long? Doesn't matter, the velocity must go up!

Pitfall 4: measuring individual velocity

The last problem I see is that managers use velocity to measure individual performance. This not only creates wrong incentives, but also might introduce conflicts within the team.

Once the team members start being evaluated by the number of points they deliver, they lose incentive to work as a team. Any activity that does not lead them to getting more points loses importance. Pair programming? Nope, because the task is assigned to another developer. Code review? Sure, as long as the other team member returns the favor. Oh, and don't expect a solid code review, just a quick "LGTM". The same with any other activity – if my performance, and therefore salary, bonus, and maybe even promotion, is based on the number of points I deliver as an individual, then I'm going to optimize for this.

What's more, this creates a competition between team members. You see, in Agile estimation, 2+2 is not necessarily equal to 4. A huge 8-point task comes with a lot of uncertainty and can take the whole week, while a 2-pointer might take just half a day. In order to maximize individual performance, everyone will try to pick the simplest tasks, leaving the more challenging ones to anyone who fails to play the performance game.

Imagine if the most experienced developer in the team, the one that helps everyone around, the one that answers tens of questions everyday, the one that maintains the high bar for code quality and who drives the technical decisions, suddenly stops and his answer to every question is "sorry, have to focus on my task". That's what using velocity to measure individual performance does to the team. And when that happens, everyone loses – the developer, the whole team, and even the process manager that came up with this idiotic idea in the first place.

The actual value of velocity

Alright, that's enough of ways to screw up the teams. Ok, just one more – never compare velocity across multiple teams. Every team has their own definition of points, hence every team has its own velocity.

Ok, now I'm done, for real. So let me explain how to actually use velocity in a way that works for everyone – the developers, the process manager, and the stakeholders.

Predictability, not productivity

First, stop trying to maximize velocity. Instead, try to minimize its volatility. If your velocity is 40 points, but you delivered 100 points 3 months ago and 10 points for 2 months afterwards, your team is unpredictable.

What it means is that process managers don't actually know how much their teams can deliver, so their promises to stakeholders can't be trusted. A team that delivers 40 points month after month is a better partner for the business than a team that sometimes delivers 100 points and sometimes 10. Business wants predictability and this is where velocity can help, if used the right way.

Combine with other metrics

Since velocity only describes the "quantity" of work done, it does not offer enough for team managers to understand how their team is doing. Maintaining the current velocity might be coming at high cost: maybe the team sacrifices quality? Maybe they ignore testing? Maybe they need to work long hours?

In order to understand the team better, managers need to look at a number of metrics and find a healthy balance between them. The exact set of metrics will vary from team to team, but at least we need to observe 3 qualities: the quantity of work, the quality, and the sustainability (or team happiness). If we ignore either of these 3, eventually we will get our team in trouble.

Make the team work together

Lastly, bring the team together. Estimating tasks by individual team members, measuring individual velocity, assigning tasks in advance – all these practices incentivize team members to focus solely on their own work. In order to build a high performing team, process managers need to change this incentive and focus people on working together. Whether it's pair programming or code reviews, or just asking each other a few questions, developers should be comfortable helping their teammates without worrying about their own velocity going down.

Listen to your team

As the last advice in this article, I strongly recommend every process manager to listen to their team. If you manage a team, ask questions about the process during 1-1s with your team members, ask them during retrospective, ask them casually over lunch. Most of the pitfalls I mentioned above are obvious to team members and they will tell you what confuses them and what doesn't work. You don't need to be an Agile expert (it's probably even better if you're not) to create a high performing team. You need to listen to your team and update your process so that it allows them and encourages them to do their best.

And here's an infographic with 4 pitfalls of velocity that you should avoid:

Team retrospective is one of the best tools in my toolkit as a manager, but it's often misused. I've seen retrospectives done just because the teams follow Scrum, without understanding the real value of this activity. In this article I explain what's the point of team retrospective, how to make most of it, and I present agenda I follow during retros I run.

I cover this topic in my last video as well, you can watch it below:

What's the point

The main point of retrospective is improvement. Retrospective allows people to introduce new ideas and point out problems, so that the team can improve their work. Without improvement, the retrospective is a waste of time. This is the most common mistake I see and it quickly leads to people becoming disappointed and disengaged.

Here's how it often goes: the team manager learns about this thing called retrospective from some article about Agile or Scrum, and decides to try it out. They organize a meeting and hype everyone about it saying "I'm there to listen to your ideas, let's make our work better! Agile, yeah!". So the team comes all excited, everyone points out some problems, brings some fresh ideas. The manager nods along, takes a photo of the wall full of post-it notes, in the end says "great meeting, very productive, we're Agile!", and then... nothing happens. A few weeks later the team comes to the retrospective meeting again, but there are fewer ideas and some of the problems mentioned sound famililar. By the time the next retro comes everyone figures out that literally nothing has changed, so they stop pointing out problems, and suggesting anything. The retrospective becomes yet another meeting that people hate to attend.

Sounds familiar? I've seen this happening many times, and sadly I also made this mistake as a manager. Over time I learned that in the key to a successful retrospective is the work that is done between one retro and another. Retro allows the team to align on what needs to change, but it does not change anything.

A recipe for good retrospective

Over time I came with a process that works quite well for me and the teams I've been managing. My goal is to keep the team motivated, and I have 3 principles that I follow as part of the process:

1. Always improve – every month (we do monthly retros) we implement at least one improvement. Usually we manage to do better, but every time I ensure there's at least one action item that gets completed, no matter how small it is.
2. Stay realistic – if something can't be changed, we acknowledge it, and move on. We don't make promises we can't keep.
3. Everyone participates – as a manager I'm often tempted to take more than I can handle. It regularly led me to failing my commitments, so now I ensure that action items are distributed among the whole team.

With these 3 principles in mind, I have a routine that I follow. Below I explain how I prepare the meeting, what tools I use, and how I keep people engaged. Note: this is what works for me and my team. You can use it as a starting point, but don't follow it blindly. Always get feedback from your team and iterate.

Preparation

Most of these steps require some time only the first time, and they can be revised in the future, so don't overthink them, don't spend too much time on them.

Frequency

If you organize retro too rarely, people will only comment on the recent problems. If you organize it too often, you'll struggle to show a meaningful improvement from one retrospective to another. I usually run retrospective once a month and I find that working quite well.

Time

Do retro towards the end of something – a month, a quarter, a sprint, a project, anything. Make it easy for people to understand what period of time they should think of. Time of the day matters if you have a team distributed across timezones, but otherwise find something that you won't have to reschedule due to other meetings. I usually do it towards the end of the month, e.g. every last Thursday of the month.

Duration

Start with 90 minutes, this should be enough. The first retro might feel pretty long, as people have a lot of things to suggest and discuss, but the next meetings will be shorter. It depends on the size of your team, how chatty people are in general, but you can always limit the number of things being discussed. Don't go over 2h, it's exhausting.

Note taking tool

Pick some digital tool. Even if your team is in the office and you use post-it notes, you'll need to digitize them. Make sure everyone can collaborate on the notes at the same time. You can start with something as simple as Google doc, though there are dedicated retro apps out there. I currently use GoRetro, which I can recommend - it's not the fanciest tool, but it does the job, and a few bugs that I reported via their chat were fixed within 2 weeks.

Retro format

When collecting ideas you need to decide how you're going to organize them. An example is "Mad-Sad-Glad" where items are added into 3 categories: what made people mad, what made them sad, and what they're glad about. There are tons of other formats that you can try, though I've noticed that a lot of them are confusing and don't add any value. I choose the simplest one I know: what "went well" and what "to improve". I also add a 3rd column to later put action items.

Note: the more fancy retro formats don't necessarily make the team better. Some of them are confusing, and some ask the wrong question. A popular "what should we stop/start/continue" format asks for solutions: "what should we stop doing?". Instead, it should ask for a problem "what didn't work well?". Once you know what didn't work well, then you can look for a solution and ask "what are we going to do about it?".

The meeting

During the meeting I have a following agenda:

• review past action items (5min)
• write down thoughts (10min)
• discuss the cards and vote (15min)
• talk about solutions and agree on new action items (20min)
• select action items to focus on and assign them (5min)

We usually start the actual meeting 5min past agreed time, so the whole thing takes us 1 hour. Here's what's happening in each part.

Review past action items (5min)

We start by looking at the action items from the past retrospectives. I keep them in one place and regularly update their status (I write more about it in the "post meeting" part of the article). This part is extremely important, because it shows the progress we're making month over month. It takes just a few minutes, but it sets the right mood. And it keeps me at check: in the end I'm accountable for my team's success, so I ensure that every month we do actually make progress.

Write down thoughts (10min)

In the next part we populate the "went well" and "to improve" columns. Everyone can write down as many cards as they want. This part might take longer, but usually everyone thinks about the retro before the meeting, so we're somewhat prepared when we come.

As this part is usually rather quiet, I like to play some casual music during the meeting. Recently I've started playing "top50" Spotify playlists, every time from a different country. It's a small, but fun thing, and it gives the meeting a bit more casual vibe.

I always ensure everyone is done writing before we move to the next step. If someone needs more time, we wait 2-3min more.

Discussing cards (15min)

The next step is to discuss all the items. This part of the conversation focuses on understanding what the author meant. We do not discuss the solutions and we don't argue that the problem doesn't exist. This part is about understanding the problem or the celebration. While discussing cards, we often notice that some thoughts are repeated, and we merge those cards together. After 15min we end up with a list of unique celebrations in the "went well" colum and a list of unique problems in the "to improve" column.

Voting

The next step is to understand which items on the lists are the most important, so that we can focus on them. Every participants gets 5 votes and we take 1-2 minutes to choose the problems we want to solve the most. The outcome of this step is that both columns have items sorted by number of votes.

Finding solutions and creating action items (20min)

Next we go through the most voted items. In the "went well" column, we acknowledge that these items made the most positive impact on the team in the last month, and we check whether any action item is needed to continue doing the good stuff.

Most of the discussion focuses on the "to improve" column. For each item we try to come up with solutions that can help solve the problem entirely or at least minimize its impact. Sometimes there's no real solution, sometimes the solution is so far away from our scope of impact that it's really impossible, but we acknowledge that. It's good to say "I know this bothers us, and we have to live with it", because at least we feel that we understand each other, and that makes it easy to stay motivated. Most of the problems have at least solution, though.

The outcome of this step is a list of action items that we plan to take.

Choose the items to focus on

In the end of the meeting we have a few action items. Sometimes there are 2-3 of them, sometimes there are 10. I ask the team what are the items we want to focus on in the next month. Usually we assign 1 item to each person, so that everyone has something to work on, but we are not overwhelmed.

The remaining items stay unassigned. I keep them as future ideas, if we ever run out of things to improve.

When we're done, I thank everyone for participating. The meeting is over, but the work just begins.

After the meeting

Immediately after the meeting I organize the notes, especially action items so that everyone in the team can easily access them and so that I can keep track of what we discussed and agreed on.

Every once a while I'll look at that list again and ask people about their progress – are we introducing the changes we agreed on? If not, why? Is anything blocking us? What can we do to get it done?

I always ensure that at least a few items on the list get done. And this is the real secret of keeping the retrospective going – I make sure that we get things done. When we come to the next retro a month later and we look at the past action items, we see a real progress. That encourages everyone to participate actively, and keeps the team motivated.

Good retrospective is simple

No fancy tools or formats will save your retrospective if you don't ensure that the team keeps improving. Sure, it's good to have a tool that allows anonymous voting, it's good to have a tool that will email a retro summary to everyone, but it's just a detail, a nuance. What you really need for the retrospective to be successful is to listen, analyze, and act. This is the only way to keep the team engaged and motivated for months or years. If you stop working on the action items, if you stop improving, the retrospective will become yet another useless agile ritual that the team attends only because they have to. Don't let that happen!

In the summer of 2020 I experienced quite severe wrist pain that limited my ability to type for extended periods of time. In order to fix it I decided to relearn typing: I switched to Ergodox EZ keyboard and Colemak layout, and I started typing with all my fingers. Here's how I decided on this approach and how it went.

Occupational hazards

While we certainly won't find software developers on the list of most dangerous professions, it doesn't mean that as programmers we're free from any health concerns. Prolonged sitting, lack of physical activity, and constant exposure to screens is more than enough to cause some long-term problems.

Having worked as a developer and manager for more than 10 years, I'm not a stranger to some of these issues. The first time I started having problems with my wrists was around 2016. Occasional pain over time grew into a larger issue, where at times I had to take an hour break from typing, because it was too painful.

Around that time I bought my first ergonomic keyboard. I replaced my beloved Apple Magic Keyboard with a split, mechanical Matias Ergo Pro. It was a wonderful keyboard, but unfortunately broke pretty quickly (and the replacement I got broke even faster...). I had it long enough to help my wrists recover though, and after a few months I moved back to Apple keyboard, and I forgot about the pain for some time.

Then in summer 2020 the pain came back, and it was as strong as ever. This time I decided not only to buy a better keyboard, but also completely change my typing habits.

The pressure (on my wrists)

Before I get to how it worked and why I thought it would work, let me explain how classic keyboards put a pressure on our wrists and how ergonomic keyboards (some better than others) ease that pain.

Our arms and wrists are very flexible, but not every position is equally comfortable or easy for them. The most natural position is when the arm is straight and the palm of the hand is turned towards the rest of the body. When we rotate our palm so that it can lie on the desk, the forearm is twisted, and you can feel that the muscle is more tense. Basically every modern keyboard (besides some very niche vertical keyboards) forces us to use the less comfortable position.

That's not all: there are 2 more dimensions in which we put a pressure on our wrists. The first one is left-right rotation. The most comfortable, resting position for our wrist when it's in straight line with the forearm, like a natural extension. Whenever we rotate the wrist, left or right, we put a pressure on it. Again, most modern keyboards require us to rotate our wrists and keep them in this uncomfortable position.

The 3rd, last dimension is up-down. The resting position here is when the hand can be loose and hang comfortably slightly below the wrist level. When we put our wrists on the table, we force our hands to be at least on the same level, if not higher, which pus a pressure on the wrist. The healthiest position when using a keyboard is to float our hands above the keyboard level. Unfortunately very few people do that, instead we put our wrists on the table or palm rest. Thick keyboards or keyboards that have a positive tilt (like the Apple keyboard on photo below) make it worse.

So we have 3 issues here. The only keyboards that offer a full solution to all the problems are vertical keyboards, but again, I've never seen anyone using them. However, it's still possible to minimize the strain on our arms with using ergonomic keyboards:

• arm twist can be eased by a tented keyboard, so that the twist is not 90°, but 70° or so (btw this is also why ergonomic mouses are way better than trackpads)
• wrist left-rigth rotation can be solved by using split keyboard, where 2 halves can be placed a few centimeters away from each other
• wrist up-down position can be improved by keeping forearms up in the air instead of resting them on the table

There are a few split, tilted keyboards available in the market:

• Matias Ergo Pro that I mentioned earlier – it has a great design and I hope the quality has improved over the last few years
• Moonlander by ZSA, producer of Ergodox
• Ultimate Hacking Keyboard
• Dygma Raise

All these keyboards partially address the 3 problems I mentioned above, but there's more to the problem than that.

I move my hands a lot

One of my issues when typing is that I move my hands a lot all over the keyboard. It's a combination of 3 different reasons.

The first one is that when I was learning how to type as a kid, I didn't learn proper practices. Instead of using all 10 fingers, I relied mostly on 5: both middle and index fingers, left thumb, and occasionally ring fingers. That still allowed me to type at 110WPM (words per minute), but it forced me to move my hands a lot.

The second reason is that I have small hands and the standard, staggered keyboard layout makes it hard for me to reach some keys, especially Y and [. Every time I need to press one of these keys I need to extend my finger and move my hand a bit.

Finally the last reason is the QWERTY layout, which is a crime against fast, comfortable typing. No, I mean it, it was specifically designed to slow down typists 150 years ago when physical typing machines could get blocked if letters next to each other were typed one after another. Yet due to that archaic limitation we're still using that keyboard layout till now.

When I decided to fix my typing habits for good, I decided to fix all these 3 problems:

• I would start typing with all my 10 fingers
• I would look for a keyboard that allows me to reach keys easier, and configure the keyboard layout
• I would replace QWERTY with something better (which is essentially any other layout out there)

In order to do that I chose Ergodox EZ as my new keyboard. It features an columnar (sometimes mistakingly called ortholinear) layout and makes it trivial to swap keycaps due to a flat keycap profile.

As you can see on the photo above, Ergodox EZ is a rather unusual keyboard - it is split, tented, has a number of additional keys, and has an ortholinear layout. You can also see that I rearranged the keycaps to that it matches the Colemak layout I decided to use. All of this makes it very unique, and yet every single feature makes this keyboard great.

So back in summer 2020 I decided to change my typing habits. In the beginning it wasn't easy, but 18 months in I can't imagine going back.

18 months of progress

The first couple of weeks were tough. I knew my typing speed would go down, but it was very frustrating. I helped myself by rearranging the keys so that I could look at the keyboard and find the key I need to press, but it still was unbearably slow. Yet, after the first month or two I started feeling comfortable with the new way of typing. I stopped looking at the keyboard, I made fewer mistakes, and most importantly my wrists stopped hurting.

It's been  around 1.5 year since I relearned how to type and I consider it a successful experiment. There are still some caveats, notably I still type slower than I used to (~90WPM vs 110WPM), and my setup is so unique that going back to other keyboards takes a moment to get used to. I still can switch to QWERTY on my laptop for short periods of time, but it's not as natural as it used to be.

Here are a few things that I did that helped me to get to where I am now:

• in the beginning I accepted I'd be significantly slower, and I forced myself to keep typing properly – working from home definitely helped since nobody could see me struggle
• I moved all the keycaps so that they now reflect the Colemak layout – it's possible because Ergodox uses flat keycaps instead of sculpted ones like a lot of other keyboards (you can see different versions of caps here: https://ergodox-ez.com/pages/our-keycaps)
• after a few weeks I started typing with closed eyes to force myself to touch type and not to look at the keyboard
• I didn't do any particular exercises, initially I tried some fast typing courses, but I quickly gave up and just continued writing code and emails
• I only switched to QWERTY when I had to write something long quite quickly and I got too frustrated with Colemak; in the first couple of weeks it was maybe once a day, later I stopped switching entirely
• I started customizing my keyboard shortcuts using Ergodox configuration tool (Oryx) to make it as easy as possible to use certain combinations

The last point deserves a bit more space – customizing keyboard shortcuts is not new, any operating system allows it. What's unique to some keyboards is that the configuration can be done on a hardware level, and that it goes beyond what's offered in Windows or MacOS.

The best example from my setup are tap vs hold. The home row keys on my keyboard as act letters when I tap them, but when I press and hold them they act as modifier keys (Cmd, Opt, Ctrl). This allows me to use common shortcuts like Cmd+C (copy) or Cmd+S (save) without having to reach keys in the most bottom row of the keyboard. Instead my shortcuts are N+C (copy) or N+S (save). It required some trial and error, but it's been a real game changer for me. Here's my current setup if you're curious.

Programming injuries

Back in 2020 I made a video about injuries and health problems that happen to programmers. In my late 20s I started having problems with my back, wrists and neck. I've realized that if I want to be able to work until retirement age, I have to change my habits. I upgraded my working environment so that I can sit/stand and type comfortably, and I started moving more - I cycle and jog regularly.

Getting a proper, ergonomic keyboard and customizing my setup so that I can keep my wrists in a relatively comfortable positon was an important step. Of coure the best solution to preventing carpal tunnel syndrome and RSI is to type less and I work on that too, but I can enjoy typing long articles without having to worry that pain in my wrists will be dictating the time of my breaks. Ergodox, while not a cheap keyboard, is worth its price. I can't remember when was the last time I felt any pain in my wrists, and that gives me hope that I'll be able to keep coding for the next couple of decades.

Discussions with some great comments from readers:

• https://news.ycombinator.com/item?id=30233383
• https://www.reddit.com/r/programming/comments/sp8hhs/i_relearned_typing_to_save_my_wrists/

Special thanks to u/jetpacktuxedo for correcting some mistakes I initially made in the article.

Earlier this year maintainers of Neovim have released version 0.5 which, among other features, allows developers to configure their editor using Lua instead of VimL. In this article I'll share a few basic rules on how to transition from one configuration to another. This is not a complete guide, but it covers almost 100% of what I needed in order to completely move away from init.vim to init.lua. In the bottom of the article you're also find a link to my config files so that you can look at them and copy if needed.

Lua in 1 minute

First, it's good to spend 10-15 minutes learning Lua in order to easily write the new config. I used the Learn X in Y minutes page, but I guess anything works. If you want to spend 1 minute instead, here you go:

-- this is a comment
num = 22 -- this global variable represents a number
local num2 = 33 -- local variable
str1 = 'this is a string'
str2 = "and so is this"
str3 = [[ and this is a string too ]]
str4 = "string " .. "concatenation"
val = true and not false -- booleans and logical operators

if str1 == 'something' then
  print("YES")
elseif str2 ~= 'is not equal' then
  print('Maybe')
else
  print('no')
end

function printText(text)
  print(text)
  return true
end

tab1 = { 'this', 'is, 'a', 'table' } -- aka array
-- tables are both arrays and dictionaries
tab2 = { also = 'this is a table' }
tab2["new_key"] = "new value"

print(tab2["also"])

require('plugins') -- will find and execute plugins.lua file

There's of course way more than that, but for me this + copying some stuff from plugins documentation was enough to write my config files.

Config basics

Ok, now onto the config. In Vim we use a number of functions that are dedicated to the editor configuration (the whole language is dedicated to it, really). In Lua we're using a general programming language and we'll use an API to interact with Neovim configuration:

• vim.cmd("set notimeout") - this is a safety net, whatever string you pass as parameter to vim.cmd will be interpreted as VimL. For multiple lines, wrap string in double brackets:

vim.cmd([[
set notimeout
set encoding=utf-8
]])

• vim.g.mapleader = "," is equivalent of let g:mapleader = ','; vim.g is a table represeting global variables
• vim.opt.encoding="utf-8" is equivalent of set encoding=utf-8; (there's also vim.o for global options, vim.wo for window options and vim.bo for buffer options, but I haven't used them)
• vim.fn is a table representing functions. You can refer to a function thisIsMyFun using vim.fn.thisIsMyFun or vim.fn["thisIsMyFun"] and you can call it using vim.fn.thisIsMyFun() or vim.fn["thisIsMyFun"]()
• vim.api is a collection of API functions. I used only one: vim.api.nvim_set_keymap that maps certain key combinations to some functions (more about it below)

Moving settings to Lua

Moving most of the settings is pretty straightforward. You just replace set x = y with vim.opt.x = "y". There are however some catches:

• pairs of boolean settings are merged into one setting, e.g. instead of set wrap and set nowrap you write vim.opt.wrap = true and vim.opt.wrap = false
• home directory problems - I had issue using ~ as a reference to home directory for some backup files etc. so instead I set HOME variable that I used by writing HOME = os.getenv("HOME")
• string concatenation uses .. operator, so to refer to my backup dir I wrote vim.opt.backupdir = HOME .. "/.vim/backup"
• double backslash - if you want to pass a special character \t to Vim, you need to write it as "\\t" in Lua

Mapping keys

The Lua API has a function to map keys to some functions. The function signature is vim.api.nvim_set_keymap(mode, keys, mapping, options), where mode refers to a letter representing editor mode ( n for normal, i for insert etc.) just like in original vim functions like nmap or imap, keys is a string representing a combination of keys, mapping is a string representing what the keys map to, and options are a table where you can pass some additional settings. Example:

vim.api.nvim_set_keymap(
  "n",
  "<leader>a",
  ":Git blame<cr>",
  { noremap = true }
 )

is equivalent of nnoremap <leader>a :Git blame<cr>.

I didn't check what are all the options that can be passed in the 4th argument, 2 that I used are noremap = true and silent = true.

I also wrote myself a few simple functions to avoid typing vim.api... every time:

function map(mode, shortcut, command)
  vim.api.nvim_set_keymap(mode, shortcut, command, { noremap = true, silent = true })
end

function nmap(shortcut, command)
  map('n', shortcut, command)
end

function imap(shortcut, command)
  map('i', shortcut, command)
end

With these functions my example above became just nmap("<leader>a", "<cmd>Git blame<cr>").

Package manager

It's very probable that you already use some package manager for Neovim, and when moving to Lua you don't need to change it, you can just wrap your whole plugin list in vim.cmd and continue using it as before.

I decided to try a new manager called Packer, which is written in Lua and requires Neovim 0.5. The installation was a bit troublesome, because I didn't know what packpath is and that Packer requires some very specific names of the directories to find packages. Anyway, I moved it to ~/.config/nvim/pack/packer/start/packer.nvim directory and it worked nicely (though I'm sure there's a better way to install it).

Besides the installation, Packer is both easy to use and has all the features I need (and a lot that I don't need). The basic config example looks like this:

return require('packer').startup(function()
  use 'wbthomason/packer.nvim'

  -- common
  use 'tpope/vim-fugitive' -- Git commands
  use { 'tpope/vim-rails', ft = "ruby" } -- only load when opening Ruby file
end)

If you need some more advanced functionality, I recommend checking the documentation.

Other plugins

There is a number of other plugins that are available for Neovim 0.5 and I found myself replacing some of plugins I had used before with the new alternatives. I won't cover them here (maybe in another post), but here are a few that you can check out:

• nvim-lspconfig together with nvim-lspinstall and lspsaga.nvim use the new, built-in LSP in Neovim and provide some useful functions. Together they allow me to easily install and use language servers (e.g. to display function documentation or jump to a definition). Together with nvim-compe (autocompletion) I use them to replace Ale and coc.vim,
• telescope.nvim replaces any search plugin you use ( ctrl-p, fzf.vim etc.)
• gitsigns replaces vim-gitgutter

There are a few more like lualine.vim that can replace powerline or airline, but I'm yet to try it out.

Summary

The whole process of moving 350 lines of init.vim to init.lua took me around 2h, including time to organize the files (Lua allows you to use multiple config files, see my example below) and excluding time to play with new plugins. I spent around 1 hour moving 90-95% of the content, and another hour solving some issues like home directory or some broken config. In the end I found the whole process rather quick and definitely rewarding, though I'm sure a lot of things can be done better. If you plan to use new capabilities of Neovim 0.5 I definitely recommend moving your config to Lua.

My config in VimL and Lua: https://github.com/arnvald/viml-to-lua

Update (February 2022):

• replaced vim.o with vim.opt (thanks to u/rainning0513) - both options will work, but vim.opt is recommended
• replaced <cmd> with : when comparing Vim with Lua to keep it consistent, since both options behave the same (again, thanks to u/rainning0513)

Update (April 2023)

• replaced compe library with nvim-cmp by the same author, and updated the config (a lot of settings have changed)
• replaced vinegar with oil.nvim, a more modern replacement that besides navigation allows creating/deleting files and supports dev icons

When interviewing with a company, it's important not only to pass the interview, but also to get as much information about the company so that we can choose the best place to work for us.

Besides obvious things like salary, paid time off, and other benefits, there are a lot of other things to pay attention to: company culture, engineering culture, product that the company is building, career opportunities, and many more.

Below you will find my video where I talk about 3 stages of the reverse interview, and later in this article you will find a list of questions you can ask. I recommend you to create your own list based on what is most important for you in a company, and use the list below as a reference.

I keep updating the list, so make sure to check out this page in the future again. By the end of 2021 I plan to have 50+ questions here.

Questions

One more tip before the list - ask questions that are as specific to the company/team/product as possible. The list below is obviously very generic, because its goal is to be a base for all companies, but you should make them more specific. Let's say you're talking to Shopify about joining their internal tools team, instead of asking "what is the plan for the next year?" you can ask "what are currently the biggest problems for developers at Shopify? How are you planning to address these problems?"

Company culture & compensation

• Question: What are working hours? Are they flexible? Are there any core hours?
  🚩 red flags: strict hours, low flexibility
• Question: Can employees work remotely? Can they work from anywhere?
  🚩 red flags: no flexiblity, or "for now from home, we will see once pandemic is over"
• Question: How often do people work overtime? Are there crunch periods? How often?
  🚩 red flags: regular overtime and crunch time, working 50-60 hours a week
• Question: How does the company keep employees motivated? Does the company measure employee satisfaction?
  🚩 red flags: no answer or very generic answer
• Question: Does company offer equity/profit sharing? How does it work?
  🚩 red flags: no equity/profit sharing or very low amount, short exercise window, unusually bad terms like most vesting happening in 3rd/4th year (that suggests company uses it as a way to keep employees who otherwise would leave)
• Question: What is the performance cycle and when do people get raises?
  🚩 red flags: lack of formal performance reviews, irregular raises (it often means people don't get a raise unless they ask, and that raises don't depend on performance)
• Question: Why is the company hiring - is it a new position or a replacement?
  🚩 red flags: a larger number of people leaving

Engineering & product

• Question: How often is the product released?
  🚩 red flags: low frequency of releases (or when talking about web app, any releases that are scheduled instead of done when feature is complete)
• Question: Can any engineer deploy production? How complex is release process?
  🚩 red flags: releases done only by team lead, any release process that requires more than 2-3 steps
• Question: Who contributes to the backlog?
  🚩 red flags: no input from engineers (it means low importance of maintenance work, quality work etc.)
• Question: What is the next major milestone for the product?
  🚩 red flags: product in maintenance mode, no roadmap, no vision
• Question: How is testing done? Do developers write tests? Are there dedicated QA engineers?
  🚩 red flags: no automated tests, no dedicated testers
• Question: Who do developers report to?
  🚩 red flags: devs reporting to non-technical people like project managers
• Question: What version of language/framework does the company use?
  🚩 red flags: relying on old versions that have reached or are about to reach end of life
• Question: Can programmers use their preferred OS and code editor?
  🚩 red flags: forcing people to use a specific OS or coding environment (especially having to connect to use some remote virtual desktops)
• Question: Do teams have flexibility in their processes?
  🚩 red flags: no flexibility, forcing people to work the same way across the whole organization, using pseudo-Agile frameworks like SAFe

Career opportunities

• Question: What are further opportunities for this role? What is the next career level?
  🚩 red flags: no specific answer, unclear answer (unless you talk to early stage startup where things are still undecided)
• Question: Does the company have career ladder and skill matrix?
  🚩 red flags: no skill matrix (for larger companies), no career ladder (for any company)
• Question: Does the company have an opportunity to progress as an individual contributor?
  🚩 red flags: no opportunity, or no clear answer (it means that management is the only way to progress beyond certain level)
• Question: How does the company help developers grow their skills?
  🚩 red flags: no answer, or something like "we have self development budget" (I'll explain that in another article, but this is red flag because the companies offer money, but not time, so people don't use that money)
• Question: Does company offer any mentorship, internal or external?
  🚩 red flags: no mentorship (except for small companies that don't have such capabilities)
• Question: What are the growth plans for the engineering team/department?
  🚩 red flags: no plans, no growth, plans to reduce team and outsource work
• Question: What does the promotion process look like?
  🚩 red flags: no process (except for small companies, promotions are quite ad-hoc there)

While the number of job opportunities for software developer skyrockets together with salaries all around the world, fresh grads and junior developers struggle to even get invited for the interviews. Why does it happen and can we do anything about it?

Tech industry has still relatively low barrier of entry, especially when compared to other well-paying jobs (think 5+ years of school for lawyers and doctors). However looking at the number of entry-level job offers on LinkedIn or other job portals, it seems that the golden days for bootcamp graduates and self-taught devs are over. Finding job as a junior has already been challenging in the last 2 years, but 2021 seems like a real bad time to be searching for the first job in tech.

In the last few years I've been heavily involved in hiring, I've worked with a number of recruiters, and I spent some collaborating with coding bootcamps. Based on my discussions and observations, I believe there are 2 main reasons behind the current situation: tragedy of the commons and COVID-19. The first one has been a problem for years, the second one has amplified the issue.

Short-term gains, long-term loss

Imagine a following situation: a group of fishermen have access to a lake with fish. The lake does not belong to any particular person, and all fishermen go there daily and catch a few fish to feed their families and sell to fellow townmen. That situation lasts for years and remains stable. Then one day one of the fishermen figures out that he can catch more fish and sell them also in other towns.

Soon other fishermen copy that model and everyone happily catches a lot of fish and makes more money. Until one day the fishermen notice that it takes them more and more time to catch fish, until one day they come home empty handed. The fishermen never considered that they need to maintain the population of fish in the lake, and because of their greed they caught all fish and left the lake empty.

This kind of situation has a name - the tragedy of the commons - and you can think about it as a multiplayer prisoner's dilemma. The dilemma is simple - you can choose to cooperate with others (limit number of fish you catch with sustainability in mind) or betray them (catch more fish for higher short-term gains). However if enough people betrays others, eventually they deplete the resource they use and everyone loses.

Tragedy of the commons is not just a thought experiment. It happens in real life, too. A fairly recent case comes from Thailand and its famous Maya bay,  featured movie "The Beach" with Leonardo di Caprio. Maya bay is a gorgeous spot located at one of the Phi Phi islands that every day used to draw thousands of tourists. Unfortunately the trip organizers were too greedy and the fragile ecosystem of the bay could not handle the number of boats and people, and the amount of trash left there. In 2018 the government of Thailand decided to close the island for recovery, and as of September 2021 it remains unavailable for tourists.

Too many betrayals among tech companies

What does all this have to do with hiring in tech? We're dealing with a similar kind of a problem:

• everyone wants to have senior developers, because they provide value
• in order to have a senior developer, we need a junior dev + a lot of time + proper training
• in order to train junior developers, company needs to invest: senior developers spend less time on product (sacrifice immediate profit) so that they can support others (with long-term gain in mind)

Here's the twist - a company might invest in a developer, but not profit from them, because once that developer gains enough experience and skills to provide value, they'll start getting other job offers, and as we know changing jobs brings higher raise than staying.

Knowing all that, companies can take different strategy depending on their circumstances:

• top companies (big tech, high growth startups) figure out that they don't really need to hire a lot of junior developers, they'll just hire the best seniors from other companies by offering them way higher compensation
• above average companies (other highly profitable enterprises) figure out they don't need to hire that many junior developers either, they'll just hire senior or mid-level devs who can't get a job at top companies
• the remaining companies know they are at the bottom of the food chain - they train developers that later find better paying options. These companies still hire juniors, because they don't have much choice, but their hiring becomes limited because there are not enough experienced devs for mentoring
• software houses are a bit of a special case - since they charge per person per day, even less experienced developers quickly become profitable for the company. It's tough to convince customers to add junior dev to the team, but often software houses just promote people very quickly

We end up in a situation where the demand for senior developers far outweighs the capacity and willingness of the companies to train juniors. Up until recently it wasn't that bad though, we were in kind of an stable situation - the number of developers needed was growing steadily, the attrition was stable, companies were offering some internships and entry-level positions, universities and bootcamps provided streams of candidates. It wasn't very easy to get the first job in the industry, but with enough effort and determination it was possible even for someone who has just started programming.

And then came the pandemic.

Nobody hires juniors anymore

After the initial shock of the pandemic, all the hiring freezes and layoffs, companies figured out where they stand and started hiring again. The bounce was strengthened by sudden spike in demand for digital services (retailers moving to on-line, booming businesses like virtual events and all kinds of deliveries), influx of cash from investors, and the need to catch-up after the hiring freeze.

Together with a lot of burnout cases and people taking longer leaves, it created a spike in need for experienced engineers, and made almost no impact (or maybe even a negative impact) on a number of entry-level roles. With all the pandemic-related challenges and opportunities, investing in junior developers is not an urgent matter.

Remote work

Working from home comes with a lot of benefits, but also plenty of challenges - how to keep the communication effective? How to ensure people keep the healthy balance between work and time off? How do we make remote teams work effectively? How do we balance between seeing each others faces while avoiding the fatigue related to constantly being on the camera?

We all still struggle with that, but  then comes another problem - how do we onboard new people? We need to deliver them the equipment, make sure they have all their need for their home office, but more importantly, we need to figure out how to share the knowledge and feel people excited about joining the company. There's no onboarding day anymore, we can't gather all new joiners in one room anymore to let them meet each other and to have a big welcome breakfast.

And then comes another complication - what if these people are fresh grads? How can we hire people for their first job while we struggle with onboarding in general? The answer is - let's put that on hold and figure it out later. Junior roles are traditionally the easiest to fill, so we can just stop for a few months and hire them later, when we are ready.

Everyone's understaffed

The 2nd effect of the pandemic is that everyone urgently needs senior developers. Gergely Orosz wrote a great blog post about it, where he explains in detail why there's a shortage of senior developers:

• sudden demand for digital products and services means companies need to scale and, in order to remain competitive, keep innovating, so they need experienced devs
• demand for developers (and limited supply) means companies have to pay more and offer more benefits
• that means it's a good time to change jobs - new job usually means a nice raise, but this time instead of 20% it can be even 50-70%
• that creates a lot of movement in the market, on one hand companies lose people, on the other bring a lot of new devs, so they need to spend a lot of time onboarding (which also puts pressure on remaining employees!)
• on the other hand people are exhausted because of the pandemic and want to take longer break; with the huge demand for developers they feel more comfortable taking a few months off (so supply is even smaller)

Once again, this affects junior developers in a negative way. Since everyone desperately needs senior devs, nobody wants to put even more pressure on their devs to help onboard and train junior devs.

And because a lot of tech companies have been playing defectors for the last few years and didn't invest enough in junior developers when they could, now everyone struggles.

The way forward

The question remains - what now? Sooner or later the situation will become more stable, but is there anything we can do to help each other? None of us alone will make a big difference, but we still can help to bring a little bit of balance in the industry, and everyone can contribute.

Why should you care?

One thing that you might be asking yourself is "why do I care?". If you're a senior developer, you technically benefit from the current situation - you can easily jump jobs and get a significant raise every time. But even then, lack of junior developers has a negative impact on your career - when there are no junior developers, there are no people to teach, so you can't improve as a mentor. There are no people who can take easier tasks, so you have to do them. Finally, the teams can't grow as fast as they could, so even if you could get promoted to lead your own team, your company only hires senior devs, and it's hard to find them.

And if you're a lead dev or a manager, you should care about building a stable situation where more experienced developers can make a bigger impact by supporting junior devs and focusing on tasks that only they can do. Without junior developers, that balance is gone.

If you're a junior dev looking for work

If you're currently looking for your first job in the tech industry, there are 2 things that you should do: expand your search and make it sustainable.

Expand: reach out to companies directly. Smaller companies could offer you a position even if they are not actively looking for junior candidates. Attend events - there are a lot of virtual, free meetups where you can chat with people. Go beyond the stack that you know, if you know React check also positions for Vue or  Angular developers.

Make it sustainable: it may take you even 12 months to find a job. Pace yourself, make a list of companies you want to write to and make it a habit to send a few applications a day. Take some days off, remember to relax, you are in it for the long term.

A year ago I made 2 videos where I talked exactly about this topic. Check them out, each of them is 5-10 minutes:

If you're an employed developer

If your company doesn't hire junior developers at all, understand why, see whether it's a rule or just nobody thought of opening junior roles. Suggest to your managert that it would help to have someone less experienced, so that you can teach them but also delegate simpler tasks that don't need someone with your experience. If not junior devs - maybe interns? In my first job I started an internship program that provided a lot of benefits over the years. If you work for a small company, it should be much easier than in a corporation!

If you are willing to help others in your free time, you can become a mentor or a coach - there are organizations out there who help people break into the industry. In the past I had a pleasure of working with a free, volunteer-based bootcamp. I helped someone to change their career, and while doing this I became a much better mentor than I used to be. It's a hard, but very rewarding work.

If you're a manager

If you're an engineering manager, you have an impact on hiring - use it! If your team is doing well, ask your manager for additional headcount and make sure it's an entry-level position. Talk to your peer and discuss with them how you can benefit from having more junior devs (I'll write about the benefits another time, stay tuned!). It's not an easy thing to do, especially if your company has practiced the parasite "only seniors here" model until now, but you can always try.

If you're an executive

If you're a VP of Engineering or CTO you are probably responsible for (or at least can influence) the hiring plans and strategy. Take junior positions into account - find a ratio that will work in your teams (1 junior in a team of 4 or 5, for example) and ensure that your company provides an inclusive, welcoming environment for people who have just started their career in software development.

Back to where we started?

The current tech boom won't last forever. Eventually things will become more stable. The world will keep innovating, the demand for developers will remain, but it won't be that urgent. In a couple of months the bounce caused by the pandemic will weaken and we'll go back to where we were before - a system where juniors don't have it easy, but where it's possible for them to get interviews, and eventually start their career in software development.

Salary negotiation is a widely covered topic - there are many videos, articles, and even books about it. As software developers we are usually in advantageous position - our industry is booming and our skills are in demand, therefore we should easily be able to negotiate salaries. Despite this, a lot of people decide not to negotiate the offers they get, because they're afraid. In this article I explain why you should always negotiate the job offer using some basic concepts from probability and game theory. And in the end of the article you'll find my video where I explain how to start negotiation when you're afraid to do it.

Expected payoff

Let's start with a concept of "expected payoff" (also known as "expected value") that I'll use throughout the whole article. Expected payoff is a single numeric value that represents what you should expect on average when participating in some activity where you have multiple options and you don't know which one is better. Using the probabilitis and values of possible outcomes, you can calculate the expected payoff of each option and choose the one that has a better potential.

Let's say our friend offers us to play a game: we'll roll a 6-side dice, and if we roll 1 or 2, we pay our friend $10. If we get 3-4 we pay our friend $5, and if we get 5 or 6 our friend pays us $20. We can use expected payoff to see whether playing this game is a good idea or not. In order to do it, we multiply the value of each possible outcome by it's probability and then we add all the values together:

• there's 1/3 chance that we'll roll 1 or 2, and we'll pay $10, so we write it as 1/3*$-10 = $-3.33
• there's 1/3 chance we'll roll 3 or 4 and we'll lose $5: 1/3*$-5 = $-1.67
• there's 1/3 chance we'll roll 5 or 6 and we'll get $20: 1/3*20 = $6.67

Now we add all the values together: $-3.33 + ($-1.67) + $6.67 = $1.67.

The outcome tells us that if we play the game on average we will win $1.67 (and if we don't play, the value is 0), so we should agree to play it. Of course the question remains why our friend wants to play such game if they're expected to lose!

On the contrary, gambling or lottery tickes have negative expected payoffs. If your chance to win $1 million is 1 in 5 million, and a single lottery ticket costs $2, then the expected payout is $-1.8. Of course people still buy those tickets hoping they'll end up millionaires, but technically it's better to invest that money in something else.

This concept of expected payoff, enriched with more advanced ideas from game theory or decision theory (that help in more uncertain situations), can be applied in multiple areas of life. Should you buy a house or rent? Should you invest in a new company? Should you choose a startup job offer over a corporate one?

Now let's have a look how it can help us understand that negotiating the job offer is a good idea.

The fear of losing the offer

One of the most common, if not the most common reason why people don't negotiate job offers is the fear that company will withdraw the offer. Candidates believe that if they ask for more, the company will decide that they don't want to have such employee. In order to understand why this makes very little sense to the company, let's use the expected payoff.

It's very hard to put some specific value to the outcomes, hiring a candidate does not directly translate to a profit, so for now I'll use abstract units. Let's say the company made an offer of $100,000 and the candidate asks for $120,000 instead. The company can't increase the salary, and therefore they have 2 options:

• rescind the offer
• tell candidate that they won't offer more, but leave the offer on the table

In terms of payoff, we can assign value 1 if the candidate joins the company, and 0 if they don't. The question is what should the company do.

The 1st option means the candidate won't join for sure, so the expected payoff of that strategy is 0. The 2nd option has some uncertainty - the candidate will probably reject the offer, but there's some chance they'll accept it. If we assume that there's 10% chance that the candidate will still join the company, then the expected payoff is 10%*1 + 90%*0 = 0.1, if we believe it's 5%, the payoff is 5%*1 + 95%*0 = 0.05 . Even if the chance is just 1%, it's still better than withdrawing the offer!

Of course reality is more complex - maybe the company has just 1 seat and they have another, equally good candidate waiting for their answer, but even in such case it's better to keep the offer on the table and tell the first candidate that the company needs an answer in 1-2 days.

Does it mean that offer withdrawal is a myth? No, it still happens, but the companies that do it either act irrationally, or they're looking for programmers that won't stand up for themselves, that will always blindly accept whatever the company offers. In other words - they're not good companies, and you're better off avoiding them.

To give more or not - company's perspective

That was a trivial example (we'll use it later though!) so now let's move to a more complex topic - should the company offer candidates more money when candidates negotiate, or not?

Let's imagine that we offer candidate $130,000 and the candidate asks for $120,000, so we can limit our options to following 3:

1. we don't offer anything more - in that case we pay candidate $130,000 and our potential payoff is 1, but the chance the candidate accepts the offer is only 40%
2. we try to meet the candidate in the middle with an offer of $140,000 - that gives us 80% that the candidate will accept the offer, but our payoff is lower, let's say 0.8 (maybe because that will make our profit from this candidate's work smaller, or because it sets a precedence where a new joiner will make more than others, etc.)
3. we agree to $150,000 - here we can expect 90% chance that the offer will be accepted, but again our payoff is a bit lower, let's say 0.6

The percentage here is not given and might vary a lot, the same with the payoff value - it's very hard to predict how a candidate joining us will impact the company. What I'm trying to model here is just a situation where we by sacrificing some value (paying higher salary) we increase the certainty that our offer will be accepted.

Now let's calculate the expected payoff for each of these 3 strategies:

1. $130,000: 40% * 1 + 60% * 0 = 0.4
2. $140,000: 80% * 0.8 + 20% * 0 = 0.64
3. $150,000: 90% * 0.6 + 10% * 0 = 0.63

In this case, negotiation is the best strategy. Again, these numbers are completely made up, the company has no details about the candidate's situation - maybe the candidate will walk away unless they get $150k, or maybe they're bluffing when they say they have other offers. What the company knows is how much they value the specific candidate and how much they can afford to pay. With this partial information they need to make a choice - and usually being flexible and offering the candidate at least some more salary can make a big difference!

You can try to model different scenarios and you'll get very different results. If it appears that it's the best not to give candidate more money, look at what made it happen. Is the certainty that candidate will accept the offer anyway very high? Or does the company assign much lower payout if they pay more than the initial offer?

To ask for more or not - candidate's perspective

Now let's look at this from the other perspective. Imagine that in our current job we make $100,000, and the company we applied for offers us $130,000. That's quite a difference, 30% bump in salary is a decent number! The question is then whether the negotiation still makes sense.

We're limit the payoff here purely to the salary and we'll omit other things like how much we like the company, whether it's a good career move etc. Then we have 3 options:

• we accept the offer - with expected payoff equal to $130,000
• we reject the offer - expected payoff is $100,000 (salary at the current company)
• we negotiate the offer and ask for $150,000 - let's try to calculate a potential payoff here.

Let's say we can expect one of 4 responses from the company:

• they withdraw the offer, which is extremely rare, so we assume there's 1% chance; this will have a payoff of $100,000 (we stay at current job)
• they keep the offer at $130,000, let's say there's 39% chance this will happen
• they raise the offer to $140,000, 40% chance
• they give us $150,000, 20% chance.

Quick calculation: $100k * 1% + $130k * 39% + $140k * 40% + $150k * 20% = $137,700, a 6% increase over original offer (and we asked for 15% more).

Again, you can try different numbers - maybe you're asking for 50% higher offer and there's a lower chance that you'll get it? But as long as you know that the chance of losing the offer is very low, you should always see that it's worth it to negotiate!

How to negotiate

As I wrote in the beginning, the topic of negotiation is covered widely in a number of videos, articles, and books. Heck, there are even companies that specialize in helping candidates to negotiate good offers from big tech companies.

The most important step though is to actually start the negotiation. In the video below I spend a few minutes talking about why you should do it, and then I explain how to start negotiating if you don't feel confident about it. Enjoy!

(the video goes live on Tuesday 2pm CEST - if you don't see it yet, check it out later!)

On this page I'm collecting a list of various resources for engineering managers. I personally tried/watched/read all of them, except when I explicitly mention that I didn't. Some of the links below might be affiliate links. I'll keep updating the list, I'm currently building a list of articles that I recommend for all EMs, so check it again in the future!

I'm always eager to try and learn new things, if you have some recommendation, ping me on 🐦Twitter or send me an ✉️email.

Books

• Accelerate: The Science of Lean Software and DevOps – probably more useful for VPs of Engineering or CTOs, this book talks about introducing Lean and DevOps practices in order to improve the efficiency of tech teams and build more resilient organizations.
• Become an Effective Software Engineering Manager (🎥 video) – great primer on engineering managers' responsibilities, it guides reader through various challenges and recommends how to approach them. Great read!
• Behind Closed Doors: Secrets of Great Management – not specific to tech industry, this book still provides useful advice for managers who want to improve their skills. It covers multiple topics like delegation, coaching, goals&objectives and more
• The Culture Map – a business book that talks about working with different cultures. The author is a consultant and shares her lessons about working with companies all over the world, but I found this content very relevant to my work as EM, since I lead international teams and I strive to understand how my team members work
• An Elegant Puzzle – a collection of tools and approaches to various problems you'll encounter in tech leadership role. I recommend it more as a reference book rather than reading cover to cover.
• Leading Snowflakes – a good intro to engineering management; Oren Ellenbogen shares 9 lessons for new EMs
• Manager's Path – a book that describes various states and challenges related to management in tech industry, from tech leads to CTOs
• Managing Humans – engineering management stories from Sillicon Valley. I gave it 4 stars on Goodreads, though I read it a long time ago and don't remember much
• Peopleware: Productive Projects and Teams – a classic book on productivity of software teams. It certainly feels dated (originally released in late 1980s), but 3rd edition was released in 2013 and the book still has a lot of value
• The Phoenix Project – a handbook in form of a novel, the book teaches about DevOps practices and looks at managing teams from perspective of inputs and outputs
• Resilient Management – a book for new managers, covers topics like setting clear expectations, and effective communication. Currently on my to-read list
• The Software Engineering Manager Interview Guide – interview preparation book by Vidal Graupera (whose blog you can also find on this page)
• The Unicorn Project – a sequel (I guess?) to "The Phoenix Project", written in similar style of a novel, this book focuses on good software development practices. In my opinion it's not nearly as good as the first book, but still an entertaining read

Tools

• EasyRetro (formerly FunRetro) – a tool for running retrospectives. I used it from 2019 to 2021 and I can recommend it
• GoRetro – alternative to EasyRetro. A bit less polished, but free. My current (mid-2022) tool of choice
• Notion – a favourite planning tool of many people, can act as a calendar, todo list, project management tool, and many more; comes with a generous free plan
• Obsidian – a knowledge base tool that allows you to connect multiple pages witn linking mechanisms. It has a growing community and an ecosystem of plug-ins
• Peakon – a web-based tool that helps to measure employee engagement, and allows employees to anonymously share feedback with their managers.
• progression.fyi – a collection of career growth frameworks and matrices, not only for developers, but also testers, project managers, and more. Very useful when you're building a propression framework for your department or company
• SourceLevel – a tool that uses various metrics to allow you to better understand your team's performance (I haven't tried it yet, and I still haven't made up my mind about how useful are such tools)
• Todoist – a powerful todo app with clients for all major operating systems; free plan available
• Typora – a desktop Markdown editor (Windows, MacOS, Linux) with a minimalist style and a lot of very well-thought features. One of my favorites, I use it to write a tech book with a lot of code examples, and to write work-related notes. Currently (August 2021) available for free

Blogs

• Camille Fournier – blog by author of "The Manager's Path". Camille doesn't write often, but when she does, it's high quality content
• Charity Majors – tons of thoughts about tech careers and leadership, a must read!
• Managers Club – interviews and articles on engineering management, by Vidal Graupera, author of "The Software Engineering Manager Interview Guide"
• Pat Kua – Pat is a coach and consultant and ex-CTO, on his blog he shares thought on the intersection of tech and management
• Pragmatic Engineer – Gergely Orosz, an experienced engineering manager shares his lessons from Uber and Microsoft, and writes about topics like career progressions and salary negotiations
• Silicon Vallye Product Group – blog focused on product managenemt, with a lot of useful content for engineering managers who closely collaborate with product teams
• Will Larson – author of "An Elegant Puzzle" and "Staff Engineer", Will shares a lot of interesting thoughts on tech careers

Newsletters

• Level Up – weekly content for leaders in tech by Pat Kua (who also writes a blog I list on this page)
• software lead weekly – newsletter by Oren Ellenbogen (author of "Leading Snowflakes" - check books section), with content about people, culture, and leadership in tech. Going strong for more than 400 editions!
• Tech Manager Weekly – another weekly newsletter with interesting articles about managing tech teams. Made by CTO Craft (check also communities section below)

Others

• CTO Craft Community – CTO Craft is a company offering coaching and workshops for technical leaders. They have a free Slack community and organize a number of events, like conferences
• Engineering Manager Slack group – a Slack community moderated by a group of volunteers
• https://github.com/ryanburgess/engineer-manager – a more comprehensive list of resources with 100+ items for engineering managers (some of them lack any description though)
• LeadDev – LeadDev is a full portal and an organization dedicated for tech leaders. It includes a ton of good articles, they organize events and workshops. Truly a great place for engineering managers.
• Plato – a place where you can find mentors (paid service) or volunteer your time to help others. I currently participate there as a mentor.
• Mentoring Club – a non-profit alternative to Plato, where you can find mentors or become a mentor. I'm currently a mentor there and you can book a session with me.

reduce (aka fold aka inject aka lfold) is a very powerful, flexible, and at the same time an unintuitive and controversial function. In this post I'll talk about what makes it both so flexible and unintuitive, and I'll present how other iteration functions like map or filter can be implemented on top of reduce. I'll use JS definition of reduce as a reference and I'll show what other languages do better in implementing this function.

Basics of reduce

reduce is a function that works on collections. It usually accepts 2 arguments: a reducer function and an optional initial value. reduce iterates over the collection, calling the reducer function for every element and passing the output of reducer to the next iteration (with one exception mentioned later). A simple example is calculating a product of all elements of the array:

// returns 2 * 4 * 6 * 8 = 384
[2,4,6,8].reduce((accumulator, el) => accumulator * el, 1);

The reducer function can accept up to 4 arguments:

• accumulator - the output of previous iteration (in the first iteration it takes the default value, or if not provided, the first element of the array)
• element - the current element of the array
• index - the index of the current element of the array
• originalArray - the whole array on which reduce is being called.

In the following example, the execution will look like this:

1st iteration: acc = 1 * 2 (output: 2)
2nd iteration: acc = 2 * 4 (output: 8)
3rd iteration: acc = 8 * 6 (output: 48)
4rd iteration: acc = 48 * 8 (output: 384)

If you want to understand it better and see more advanced examples, check the tutorial I recorded:

Use cases

reduce has traditionally been a part of functional languages, where it acts as kind of equivalent of for loops. It became more common thanks to a MapReduce framework which allows to easily parallelize operations that aggregate some data. MapReduce splits the work to be done in 2 parts - map part performs some kind of operation on each piece of data (this part can be done in parallel) and reduce then gathers all the output from map and combines the filan result (this part is done sequentially).

Let's say we want to count number of occurences of each word in a piece of text. We can split the text into sentences, and fo each sentence we can calculate number of occurences of each word in parallel. Then we end up with multiple dictionaries, let's say:

{ "dog": 2, "is": 2, "animal": 1, "and": 1, "mammal": 1},
{ "fish": 1, "is": 1, "animal": 1, "too": 1}

Then reduce function can merge these 2 dictionaries and calculate final output:

{ "dog": 2, "is": 3, "animal": 2, "and": 1, "mammal": 1, "fish": 1, "too": 1 }

Interestingly, reduce does not need map to achieve the result above - it's only needed in order to have the first part run in parallel.

Another common use case is to calculate some number that's based on a list of numbers. A good example is sum of squares that has a number of uses in mathematics like in linear regression.

I personally often use reduce in order to transform one dictionary into another (e.g. I might need to normalize keys, or update values). This is not possible in JavaScript though - I explain it a bit later in the article.

The controversy

For a number of reasons, reduce is a controversial function among programmers. In JS it gets quite a bad rep, like in the widely retweeted example below:

It's not the only example, though. In Python, reduce was removed from the standard library and moved to functools library. It's still shipped as part of the Python language distribution, but in order to use it, you need to explicitly import it.

There are a number of reasons why reduce gets a bad reputation, the main of them being: for every use of reduce there's at least one more intuitive and more readable alternative.

For loops and other options

First argument for not using reduce is that many languages (mainly imperative/OO) there are always more idiomatic and intuitive ways to write code than useing reduce. The main solution is to use for loop, forEach function, or some kind of equivalent. Let's take the example from the previous section:

[2,4,6,8].reduce((accumulator, el) => accumulator * el, 1);

Another way to write is

const product = 1;
for (const el in [2,4,6,8]) {
    product *= el;
}

For programmers coming from other imperative languages, the latter version certainly feels more familiar. Is it clearly better though? I'm not so sure.

Readability

The second argument is quite similar, but focuses on reduce function itself - a lot of people say the function is hard to read. I partially agree with this. Most of the time I have little problem understanding what is the goal of reduce just by having a quick look, but because it can return anything, it's not as meaningful and intuitive as map or filter. What's more, if you want to use reduce in multiple programming languages, you'll have to remember that each of them has a different number and order of arguments!

There's one more thing that adds to the problem - the initial value, which is an optional parameter in reduce and which changes a lot about how the function works. If you have a collection of 10 elements, you can expect that it'll trigger 10 iterations, however if you don't pass the initial value to the function, there'll be only 9 iterations. That's because the first element of the collection will become the initial value. In a lot of cases, like when calculating a sum or a product, it doesn't matter, but when you want to calculate sum of squares, that missing initial value will break the function!

function sumSquares(ary) {
    return ary.reduce((acc, el) => acc + el * el);
}

sumSquares([1,2,3,4]); // => 30, works!
sumSquares([4,3,2,1]); // => 18, broken!

Limitations

The last reason applies to some specific langauges, for example JavaScript - reduce was added to JS as a half-baked thing, working only on arrays. The same function in other languages can be used on other types of collections. In Ruby as long as a class includes the Enumerable module, it gets reduce function. In Python, where reduce is used very rarely, you still can use it with dictionaries. I believe reduce would be way more useful in JavaScript if only it was possible to call it on other types of collections.

Write everything in reduce!

While I agree with the arguments I presented above, I still believe that understanding reduce can be very helpful, especially if you ever consider learning functional languages. It's really a powerful function. Actually, reduce is so flexible, that a lot of collection functions can be rewritten using reduce. Let's try it out!

Warning: don't try to do it in your apps. The original implementations of the functions below are certainly better (and probably much, much faster).

forEach

First, something easy: forEach is a reduce that calls a passed callback and does not return any value.

function foreach(array, cb) {
    array.reduce((_acc, el) => cb(el));
}

map

map is reduce where we start with empty array and in every iteration we add result of the callback function to the accumulator.

function map(array, cb) {
    return array.reduce((acc, el) => [...acc, cb(el)], []);
}

A slightly more readable (and faster, I guess) version, with 2 statements, would look like this:

function map(array, cb) {
    return array.reduce((acc, el) => {
        acc.push(cb(el));
        return acc;
    }
}

flatMap

This one's quite complicated! flatMap behaves similarly to map except that it always returns a flat (1-dimensional) array. If the provided callback returns an array, map returns an array of arrays, while flatMap, as the name suggests, flattens the output. It could be implemented this way:

function flatMap(array, cb) {
    return array.reduce((acc, el) => [...acc, ...cb(el)], []);
}

However, if the cb does not return an array (we can't guarantee it does), we need to add something more. There are a few different ways to deal with it, the most trivial is to just flatten the outer array. It's not a pretty solution (and oh it is SO slow), but it will do.

function flatMap(array, cb) {
    return array.reduce((acc, el) => [...acc, ...cb(el)].flatten(), []);
}

filter

Next, filter returns elemets of orignal array, but only those that meet the provided expectation (read: where cb(el) returns truthy value). First, let me implement it using 2 statements to make it easier to read.

function filter(array, cb) {
    return array.reduce((acc, el) => {
        if (cb(el)) acc.push(el);
        return acc;
    }, []);
 }

Now the same can be rewritten with a single statement, though it's less intuitive.

function filter(array, cb) {
    return array.reduce((acc, el) => {
        return cb(el) ? [...acc, el] : acc;
    }, []);
 }

some

some returns true if callback function returns true (or any truthy value) for any of the elements in the array. It can be written in pseudocode as cb(array[0]) || cb(array[1]) || ... || cb(array[n-1]). In order to implement it with reduce I'll be carrying on the boolean value over each iteration.

function some(array, cb) {
    return array.reduce((acc, el) => acc || Boolean(cb(el)), false);
}

every

every is a sibling function to some and returns true if the callback function returns true for every element of the array. It can be written as fun(array[0]) && fun(array[1]) && ... && fun(array[n-1]). Similarly I'll carry a boolean value as acc.

function every(array, cb) {
    return array.reduce((acc, el) => acc && Boolean(cb(el)), true);
}

includes

includes could actually be implemented using some. For the sake of consistency I'll just keep using the reduce directly though. In this case we don't have a callback to use, instead we need to check if any element is equal to provided value.

function includes(array, value) {
    return array.reduce((acc, el) => acc && (el === value), false);
}

As a side note, the 3 functions above are examples where using reduce introduces a performance penalty (they'll iterate over the whole array even if they could stop earlier). One more reason not to use this code in any serious application.

find

find returns the first element that meets a criteria specified by the callback function. In terms of implementation, it's similar to some with a twist. Just like with some we're going to pass a certain falsy value and as soon as it becomes truthy, we're going to pass it till the end of the iteration process. The twist is that the value we need to pass is not the output of the callback function, but the element on which the function is called.

function find(array, cb) {
    return array.reduce((acc, el) => {
        if (acc) return acc;
        if (cb(el)) return el;
    }, null);
}

Earlier in this post I said I'd try to write the reduce with only a single expression. It's possible in this case as well, though just as before it's harder to understand:

function find(array, cb) {
    return array.reduce((acc, el) => acc || (cb(el) && el)), null);
}

The cb(el) && el part will return false if the element does not meet provided requirement, or it will return the value of el if it does. Then the first part, acc || ...will either return acc (output of previous iteration), unless it's a falsy value, in which case it'll return the 2nd part explained above.

findIndex

findIndex initially seemed more challenging to implement, because somehow I need to keep track of the index together with the element. Then I remembered that the reducer function takes 4 arguments, and not only 2! The 3rd argument is the current index, and 4th one is the array on which the reduce is called (I'm still thinking how to use it in practice). So findIndex will be almost identical to find.

function findIndex(array, cb) {
    array.reduce((acc, el, i) => {
        if (acc) return acc;
        if (cb(el)) return i;
    }, null);
}

lastIndexOf

lastIndexOf is almost the same, except that first we check whether the current element meets the expectation, and only if it doesn't, then we return the last on that did. In short: we swap the order.

function lastIndexOf(array, cb) {
    array.reduce((acc, el, i) => {
        if (cb(el)) return i;
        if (acc) return acc;
    }, null);
}

Similarly to find, the findIndex and lastIndexOf functions (why isn't it called findLastIndex by the way? and why there's no findLast function?) could be rewritten using a single expression, the only difference is the order and the logical operators used.

Can reduce do everything?

Looking at the list of array functions in JS and I was wondering if there's anything that can't be implemented with reduce. Initially I had 3 ideas:

1. Functions that modify the original array - reduce comes from languages with immutable data structures, so modifying original array (with functions like copyWithin) was a long shot, but because the reducer accepts original array as a parameter, it is possible (I am 99.99% sure it's always bad idea, though - don't do it at home!)
2. Sorting - ok, when that idea came to my mind I thought it was really stupid, but maybe it's possible to implement some kind of bubble sort with reduce? Well, it seems I was not the only person who wondered about it!
3. Finally, I found something - Array class has methods like keys and entries, and those functions return iterators. I tried to implement them with reduce, but I failed miserably, so I assume it can't be done (correct me if I'm wrong!).

What's the point?

This was a fun exercise, but my point here is that each function has its place. reduce gets a lot of bad rep in JS and for good reasons. It's limiting yet overcomplicated and I still don't remember the order of parameters in reducer, although I used it a number of times. Still, it's good to understand it, so that you can use it from time to time.

Oh, and of course - check out other languages where reduce work also for dictionaries, sets, or other collection types. Languages like Elixir, Haskell or Ruby make reduce more powerful and intuitive at the same time!

Engineering manager role comes in a lot of flavours and the interview processes for this role at various companies reflect that differences. In this video I'm talking about what kind of interviews and questions you can expect, and below the video you can find a list of 70+ questions that you might hear during different interview rounds.

Video

🎥 timeline:

0:00 developer interview vs manager interview
0:59 programming interview
02:20 system design interview
03:36 system design - how to prepare
05:30 team management interview
07:25 stakeholder management interview
08:20 management interview - how to prepare
11:05 general interview tips

Questions

Leadership

• what is your leadership style?
• what is the role of engineering manager?
• what does it mean to be a successful engineering manager?
• how do you define a successful team?
• how do you define an individual success?
• have you ever had a very urgent situation when you had to make a quick decision? what was that and what did you do?
• have you ever made a bad decision that affected your team? how did you resolve the situation?
• who's the best leader you've ever worked with? what have you learned from them?
• who's your role model? why them?
• how do you motivate your team?
• how do you handle team members that lack motivation to grow?

Team management

• how do you onboard a new team member?
• have you ever had a person that was difficult to work with? what did you do about it?
• have you ever had to let go a member of your team?
• have you ever had a conflict between your team members? how did you deal with it?
• have you ever built a team from scratch? how does it differ from taking a lead of existing team?
• how do you keep track of ongoing work in your team?
• do you do 1-1 meetings with your team members? how do you do it?
• have you ever managed people who are way more experienced than you? how did you help them succeed in their roles?
• have you ever managed a team budget? how did you keep track of it?
• have you ever made a decision that your team didn't like?
• have you ever managed a fully distributed team? how did you ensure the team was successful?
• how do you stay in touch with your team when working remotely?

Performance

• how do you measure individual performance in your team?
• how do you measure performance of your team? what kind of metrics do you use?
• what kinds of performance review process are you familiar with? which of them works the best in your opinion?
• have you ever had a very high performing team? what made the team work so well together?
• have you ever had an underperforming team member? what did you do about it?
• have you ever had to put someone on PIP?
• have you ever promoted a team member? how did you help them progress?
• have you ever denied someone a promotion? how did you explain it to them?
• have you ever turned a low performing team member into a high performing one?
• how do you pass a message about a low score in performance review? how do you keep a low performing employee motivated?
• have you ever used or created a skill competency matrix?

Hiring

• what's engineering manager role in hiring?
• who do you involve in hiring process?
• who do you involve in making hiring decisions?
• have you ever made a bad hire? what did you learn from it?
• have you ever negotiated an offer with a candidate?
• have you ever decided not to hire a candidate when other people were very positive about that candidate?
• what do you pay attention to when hiring?
• how do you choose the best canidate when you have many good candidates and just one seat?

Delivery & processes

• have you ever had to manage multiple urgent tasks? how did you do that?
• have you ever dealt with project that missed its deadline? what did you do about it?
• have you ever led a project that failed? what made it fail? what did you learn from it?
• have you ever dealt with suddenly changing requirements? what did you do about it?
• are you familiar with agile methodologies?
• have you ever worked with Scrum/Kanban/SAFe etc? How do you understand your role in that methodology?
• have you ever had to manage unclear product requirements? how did you do it?
• have you ever taken a struggling project and turned it around? how did you it?
• have you ever introduce a significant change in processes? how did you convince others to implement it?

Stakeholder management

• have you ever worked with UX designers/Technical Writers/testers/product managers? How did you ensure that you worked well together?
• how do you imagine a collaboration between engineering manager and product manager?
• have you ever had a conflict between you and some stakeholder? how did you solve it?
• how do you ensure that your team and other stakeholders are on the same page?
• how do you pass the feedback from your team to other stakeholders?
• how do you work with stakeholders when project is delayed?
• who were the most important stakeholders in your previous roles?

Technology

• do you have experience with Python/JS/Java etc.? Have you used Postgres/Jest/Hadoop etc.?
• what's your opinion on Postgres vs MySQL, Hadoop vs Spark etc?
• do you have experience with e-commerce/cybersecurity/logistics/other business domains?
• have you ever dealt with legacy software? how did you balance between technical debt and feature work?
• how do you introduce a new tool to the company?
• how do you choose whether to buy some software or to build it yourself?
• what's the software you wrote that you're most proud of?
• what's the last technology you've learned and why did you choose it?
• how do you promote best programming practices in your team?
• how do you stay up to date with ever changing technologies?
• what's your favorite programming language/framework and why this one?
• how do you implement monitoring in order to minimize downtime?

If you're a JavaScript developer who's interested in topics like machine learning, data science, or even if you want to learn some new tool to broaden your experience, there's a high chance that you are trying out (or going to try out) Python. While both languages are quite similar, there are a bunch of surprising differences that can confuse you. In this article I list a couple of such gotchas together with examples and explanations which will help you understand why both languages behave differently.

And if you are looking for a new language to learn as a JavaScript developer, check out my video where I talk about how to choose your next language.

Anonymous functions are very limited

First on the list: anonymous functions aka closures aka lambdas. They're a very common tool in JavaScript. To be fair they'r nothing special, just ad-hoc created functions that do not have any specific name. Any time you write array.map((el) => doSomething(el)) you write an anonymous function. Heck, to be fair probably 99% of JS functions you write these days are technically  anonymous. Look at this:

const myFun = () => alert("I have no name!");
const otherFun = function() { alert("Me neither!"); }

These 2 functions are anonymous. You can refer to them via variable to which they're assigned, but if that variable ever changes the value, you can't find the function again, it's lost. In order to have a "real" named function, you need to do this:

function myFun() { alert("I have a name"); }

class MyClass {
    otherFun() { alert("Me too!"); }
}

There are certain other differences between named and anonymous functions in JS, but in general they're very similar and you can easily use either of them most of the time. However, in Python the situation is different. While the language supports both named and anonymous functions, the latter are very limited: they can consist of only one expression (essentially they can do one operation). To show you an example:

fun1 = lambda : print("works!")
fun2 = lambda a, b : a + b

wrongFun = lambda: # this will throw invalid syntax error!
    a = 1
    b = 2
    return a + b

This means that the following piece of code is impossible to write using lambdas in Python:

makeRequest().then((result) => {
    logOutput(result.data);
    saveContent(result.data["content"]);
    return result;
}); 

Why is there such limitation? As Python's creator himself explains:

I find any solution unacceptable that embeds an indentation-based block in the middle of an expression.  Since I find alternative syntax for statement grouping (e.g. braces or begin/end keywords) equally unacceptable, this pretty much makes a multi-line lambda an unsolvable puzzle.

Basically in order to allow multi-line anonymous functions, a single expression would need to respect the tab-based indendation rules (currently it doesn't) or use another block separators (like {} in JS). Guido van Rossum, creator of the language, rejected both these ideas.

What's the solution then? Well, simply give the function a name! To be fair it's not that much of a deal, rather an inconvenience, but also a gotcha, something that I didn't expect when I first learned Python.

Expressions vs statements

Related to the previous point are differences between statements and expressions in both languages. If you're not sure what are these two, a brief explanation is that expression is something that produces a value, while statement is just a piece of code that performs something, but it does not return any value. In other words, you can ask yourself: can I assign it to a variable? If yes, it's an expression, otherwise it's a statement. For example:

const a = 3; // 3 is an expression
const b = a + 12; // arithmetic operations are expressions
const c = (z = 10); // (z = 10) is also an expression
const d = (if (a > 2) { 7 } else { 2 }); // this won't work! if is a statement

function myFun() { alert("alert"); }
const e = myFun(); // calling a function is an expression

Ok, so what's the issue? Well, the issue is that an expression in JavaScript might not be an expression in Python! For example, creating a function:

const something = () => 8;
const sthElse = function namedFun() { return 7; } 

This code is perfectly valid in JavaScript (even if it's not common to assign named functions to variables). What about Python?

something = lambda : 8; # this code is valid
sthElse = def namedFun(): return 7; # this code will crash!

In this case the 2nd example does not work, because defining a named function in Python is a statement, not an expression. Similarly with assignments let a = 10 in JS returns 10, while in Python a = 10 returns nothing, it does not produce any value.

I am not sure why Python function definition is a statement. A possible explanation is that on one hand indented lines inside an expression do not create a new scope (which is logical, why would they) and on the other hand function definition must create a new scope, therefore a function definition can't be an expression. That's just my theory though, maybe it was decided a priori that definition is a statement and that's it.

I can't think of any more diffecentec between expressions and statements in Python vs JavaScript, but if you're interested in how it looks in other languages you can check Ruby, where essentially everything is an expression (including if, for etc).

Tricky default parameters

Default values for function arguments are a feature so obvious that it is rarely ever mentioned besides basic tutorials. It's easy, if a value is not passed to your function explicitly, instead of throwing an error, you just give it a certain, hardcoded value.

const processJob = (name, args, delay = 0) {
    Job.fetchByName(name).startIn(delay).execute(args)
}

processJob("createUser", {name: "Ian"}, 60) // run in 60sec
processJob("createUses", {name: "Ion"}) // run now

In Python, default values have a catch though - instead of being evaluated every time a function is called, default values are evaluated only once. Why does it matter? Because if you decide to modify the argument inside your function, it will not be brought back to its previous state! Let me show it using an example. First, JavaScript:

const addOne = (ary = []) => {
    ary.append(1);
    return ary;
}

addOne([3,2]); // returns [3,2,1]
addOne([3,2]); // returns [3,2,1] again

addOne(); // returns [1]
addOne(); // returns [1] again, this is crucial here    

Now lets compare it with Python:

def addOne(ary=[]):
    ary.append(1)
    return ary
    
addOne([3,2]) # returns [3,2,1]
addOne([3,2]) # returns [3,2,1]

again addOne() # returns [1]
addOne() # BOOM! returns [1,1]
addOne() # and now returns [1,1,1]

See the difference? The default argument is always the same array. Unlike most of the languages I know, Python does not recreate this object every time. The common solution to this problem is unfortunately a rather ugly idiom:

 def addOne(ary=None):
     if ary is None:
         ary = []

     ary.append(1)
     return ary      

Note that this difference applies only to complex data types like arrays, dictionaries etc. If your argument is a string or a number, the default value will not change from one call to another, so you can safely use it. However if you want to have an array or dictionary by default, you need to pass None as a default value.

Nothing is private!

Alright, there are way more examples, but let's not turn this article into a compedium. Let me just mention one last gotcha in this post - privacy... or rather lack of it. As you probably know, JavaScript does not have an explicit concept of private methods or variables. I mean whatever you put in a class can technically be accessed outside of it. There is a proposal to turn add private methods and fields with kind of an unusual syntax, but for now it's not there yet.

Still, there are ways to make some stuff kind of private in JS. For example you can write a set of functions and variables and export only a few of them:

const x = 12;
const y = 10;

const pubFun = () => console.log('public');
const priFun = () => console.log('private');

export { x, pubFun };

And of course you can use Typescript, which has a full Java-ish (C++-ish?) set of function/field access control.

On the other hand, in Python essentially everything is accessible - there is no built-in protection mechanism. There is no export keyword - everything can be imported from any module, everything can be accessed in any class. Python promotes an "unwritten agreement" approach, where underscores used as prefixes indicate that the function or variable should not be used outside of its original class or module:

var1 = 0; # go ahead, use it whenever you need
_var2 = 0; # should not be used outside of its class/module
__var3 = 0; # DEFINITELY should not be touched

To be precise, Python does a little bit to discourage use of functions with double underscore (read more about it here), but you still can use "private" functions whenever you want, wherever you want.

More gotchas?

Programming languages are very complex tools and therefore have a lot of surprises. Sometimes they're obvious and we learn them very early and sometimes it takes months or even years to notice a behavior that surprises us. I'm sure Python has more getchas for JS developers. If you have some other examples, tell me!

And if you're curious about why Python became such a popular language, check out my other post where I compare it to other dynamic, scripting languages.

I've recently read a rather popular blog post about cancelling stand-ups and how it affected team's productivity. While I find the topic extremely interesting and I like experimenting with changing processes, the post lacked some long-term conclusion (hopefully a 2nd part is coming in a few months!). I thought I could share my experience with making stand-ups shorter and more effective. I do not have any hard numbers to show, what I have is experience and some rough estimations from 3 companies I worked with where I participated in daily stand-ups.

15 minutes or 1 hour?

In the 1st company, I was a new employee and I had little say about team processes, so I followed the by-the-book style that the team implemented.

Every morning I arrived at work around 9:30am, I grabbed a cup of coffee, and in 5-10 minutes I was at my desk ready to work. However, at 10am I had a stand-up, so I spent the 20min I had on replying to e-mails and chatting with my coworkers.

Stand-up was in a separate room, where the whole team met at 9:55am so that we could connect with a few people who worked in another office. Ignoring random audio/video problems, we usually wrapped things up in 20 minutes, with occasional 10min follow-up after the call. Sometimes discussion took a bit longer, but usually I was back at my desk by 10:30am, an hour after coming to work.

I guess this is not exactly how stand-up looks like in your "Scrum for beginners" manual, but that's how it worked in practice, and trust me, that company is not an outlier. It worked quite well for some people, especially those who came to work just before 10am, while I found the lost hour a bit frustrating. I tried to change it, but everyone else felt ok with the way things were, so we kept it the way it was.

The total time I spent on stand-ups was around 30min a day, so 2.5h a week (5h if counting the 30min a day that were kind of wasted, but who knows, maybe I would be wasting them anyway).

How to scale a stand-up?

In the 2nd company I was a team lead and we had a classic, quick stand-up. Next to our desks, all people standing in a circle, one by one saying what they did, what they are doing and what's blocking them. Anything to discuss? Do it in smaller groups right after the stand-up. It usually took 1min per person, 10, maximum 15min in total.

This was going well for a while, but over time it became more difficult to manage. First, as we were approaching 15 developers and we split into 2 teams, we needed a way to split the stand-ups as well, but how to do it when we all sit together? First goes one group and then another? That felt awkward, as did having everyone go to a meeting room for 10 minutes.

The 2nd challenge was that we expanded our team to a new office in another country. One of the things I wanted to prevent was separating all work and rituals by location - this alienates people working from new office and makes them feel less attached to the company. We considered having a call every morning, with 2 groups standing in front of the camera in their respective offices. It's not a bad solution, but having experienced it in the past I knew how awkward it feels, and how making sure the connection works can take longer than the meeting itself.

Good morning! Yesterday I...

So what we did was to remove stand-ups. Or rather, we removed stand-up as a meeting, while keeping the update part. We moved it to Slack, so that every morning, every team member was supposed to write a short message when they started working. A short "good morning team!" followed by 3 bullet points. This provided a few benefits:

• scaling - each team has their own channel where they write the message
• persistence - messages stay on Slack, so it's always easy to get back to something that I was supposed to follow-up on. Also, it made it possible to add links to tasks or documentation, and discussing any blockers was easy - just reply to the update message
• flexibility - you come to work, you sit down, you check what you've got to do today and you write "Good morning everyone! Here's my update...". It's the first thing you do, and then you can start working. It takes 2-3 minutes and you don't need to wait for anyone. You can read other updates later when everyone's in, and if someone needs your help, they'll tag you.

So in this case we didn't really save any significant amount of time (not that there was a lot of time to save in the first place). Writing and reading all the messages could even take a bit more than before. But we got a lot in exchange - and we never looked back.

Rinse, repeat

This experience helped me in the 3rd company. When I joined it, stand-ups were happening over a call (around half of the team was remote) and usually took around 30-40 minutes. Yes, you're right, that's way too long. 30-40 minutes * 5 days means 2.5-3 hours a week spent by every team member on what's supposed to be a quick update.

I quickly realized the issue was that while a few people had short and concise updates, others were taking their time and often started longer discussions. I took various attempts to change it - tried to limit stand-up time to 15 minutes, reminding people to discuss other topics after the call in smaller group, etc. but I was not satisfied with the results. Finally I got a small win - we introduced Fridays without meetings and that meant no stand-up either. In order to stay in touch about the progress and blockers, we decided to just write updates on chat. What started as once a week initiative, after a couple of weeks turned into a daily practice, we moved to entirely asynchronous stand-ups and again we never looked back.

As I have mentioned before, the amount of time we saved was around 3 hours per person every week. For me, this was huge!

Not perfect

One thing I'm really happy about is that in both cases where I managed to introduce the change, after a while it gained essentially unanimous support - everyone told me they preferred the asynchronous updates.

There are a few aspects though in which this idea could be improved, and a few in which it will always be worse than a classic meeting. Seeing and hearing your peers is something that can't be replaced. Reading messages removes that human touch, it strips us of things that make us feel that we're talking to real people. In my case this wasn't a big issue as we had enough other calls each week anyway not to miss each other, but it might not be the case in every company.

Missing messages is another challenge - if you come to work first and write your message, then you need to make sure that later you read messages by other people who start work later. If people don't do that, then what's even the point of sending updates? I struggled a bit to follow all messages everyday, something that didn't happen during classic stand-up. This problem can be solved with a dedicated tool that will remind you to send a message each morning, and will compile all updates into one longer message when all team members provide their updates. Alternatively you can set a reminder to read all the messages at some point during day, when you expect them all to be there.

Throw out the book

One of the most common problems I have when trying to improve processes are people who specialize in certain system or methodology (often certified in it), who tell me that's not how things should be done. Whether they're fans of Kanban, Scrum or something else, I often hear "we should not do it, this is against Scrum guides".

I believe such attitude is against the principles of good team work. Whenever something doesn't work, I try to change it. If it gets better, I keep the good parts and keep improving. If it gets worse, I learn the lesson and change the direction.

Me and my teams in the past changed the daily stand-ups, because they didn't work for us. We wanted to stay updated, so we kept that part, and we improved the rest. I don't know if that's going to work in your team too, maybe not - only you can try it out and see if your team likes it. So go ahead, keep the good parts, throw out the rest of the book, try to make your way of work better. Good luck!

Credits

Burning book photo by Fred Kearney on Unsplash
Paddleboard photo by Krzysztof Kowalik on Unsplash
Scale photo by Bartosz Kwitkowski on Unsplash
Hand washing photo by Rizal Hilman on Unsplash

This post was inspired by "How Dev Bootcamps Are Failing Their Students" by Tyler Hawkins (and yeah, I started writing it 3 months ago, something something about unfinished projects).

Software development bootcamps have passed their hype stage and now they're somewhere around the trough of disillusionment and the plateau of productivity. New bootcamps keep opening, but it's harder to find people who believe the overpromising "become a professional developer in 10 weeks" slogans. As long as the demand for software developers grows though (and it almost certainly will when economy starts recovering), bootcamps are here to stay. We, lead developers and engineering managers, need to learn how to make the best of it.

The rise of the bootcamps

According to Wikipedia, the first programming bootcamps were created around 2011. While they were met with a solid dose of skepticism, it is impossible to ignore their impact on the software development industry. Yes, it has always been possible to become a programmer without a degree in CS, but let's be honest - it was a difficult path, that required a lot of discipline, determination and spare time. Suddenly coding bootcamps changed it - they promised to turn newbies into professionals within a few months or even weeks. Too good to be true? Certainly a little bit, but hey, for thousands of people it actually worked!

The success of bootcamps and the chance to join tech companies after a few weeks of training was possible due to 3 major trends happening at the same time:

• the rise of SaaS powered by faster internet, cheaper hosting, AJAX, and new easy-to-start frameworks like Django or Ruby on Rails
• grow of tech companies, including tons of new startups supported by VC money
• boost in process of digitalization across multiple industries

These 3 things led to a huge increase in demand for software developers. I'm not kidding here - when I attended some tech conferences in 2011, I heard "by the way, we're hiring" in every single presentation. Universities, which are traditionally the source of workforce in intellectual professions, could not keep up with this need - even if they suddenly doubled the number of new CS students, we would have to wait at least 3-4 years to see the effects.

So on one hand there were tons of companies that had enough resources to hire more developers, on the other there were people who were ready to switch careers, lured by a promise of quick training and high salaries. Boom! Welcome to a software bootcamp - a badly needed solution for the demand problem.

What's the goal of a bootcamp?

This context is important to understand why bootcamps' curriculums look the way they look. Bootcamps were never meant to replace a university degree. They're not "CS in a nutshell", I wouldn't be surprised if you could complete a bootcamp without hearing the term "computer science" at all.

The goal of the bootcamp is not to teach you CS or even to teach you software engineering. The primary goal of bootcamps (besides making money of course) is to help their clients to land their first job as a developer/designer/data scientist/whatever. This is the primary metric bootcamps use to show that they're a good investment, this is the primary selling point - you give us $$$ and put a lot of work for a couple of months, and we'll help you to land a job in a booming, well-paying industry.

This is a critical thing to remember - bootcamps optimize for job interviews, therefore their curriculums reflect the desired skills, the technologies that are in demand and the questions that are commonly asked during interviews (BTW this is how the tech companies can shape the curriculums of bootcamps - by changing the way we do interviews).

Fresh grads and shaping knowledge

Because of the very limited scope that's taught at bootcamps, we - team leaders and managers - need to adjust our ways of mentoring and coaching new employees to help them progress and become productive.

When we work with fresh university graduates, they posses some knowledge that's ready to be used, but more importantly, they have spent a few years studying different topics that can be useful in software development, so they have much more knowledge that can be "unlocked" over time. Bootcamp graduates naturally lack this knowledge, simply because they've never studied these subjects. They're more like "what you see is what you get" - in order to make progress they can't just access the information they have learned in the past, they need to get it first (to be fair they often have a lot of knowledge of other topics, especially if they have previously worked in other industries - part of this knowledge can be translated into faster improvement of programming skills).

I like to compare this to pottery and clay - university grads accumulated a lot of material over years to start turning into practical skills quite quickly, bootcamp grads need to get this material first, because the few weeks they spent studying is just not enough to get enough of it.

Practical mentoring

Finally we can get to the point where I can talk about some practice and how to help bootcamp graduates to progress quickly.

The first part of the process is to make sure that our new colleague is aware of the importance of CS fundamentals. They need to know how learning basics of data structures and programming paradigms makes it easier to learn other concepts. At the same same time they shouldn't be ashamed they don't know these things, we need to ensure them that gaps are expected and we're there to help. Often people realize that there are many things they do not know yet, but it's still worth to talk about it.

The second step is directing them to the right resources. While you can draw up a plan of what they should learn step by step, I believe what works better is to follow the natural flow - if your team member needs to do some database work, point them to the right book about SQL and to someone in the team who can help them if they get stuck. If they need to improve performance of some script, tell them about how to profile applications and how they can find better algorithms that can replace some brute force solutions. Don't try to force them to follow classic CS curriculum - knowing how operating systems work is great, but it's not the most urgent thing to learn.

Third part is the pace. A job in the new industry is a stressful thing. No matter how hard we try, our new team members will have doubts, will have moments when they want to quit and give up on programming. You might not see it, they might not say it aloud, but there's a high chance that they will struggle and will work hard every night to keep up with their colleagues. That's why the pace is important. Learn 1-2 new things at a time, don't try to understand everything at once, relax and get good sleep everyday, keep slower but consistent pace - we need to repeat these words to reassure the new employees that they're on the right track.

Finally, something that I shouldn't have to say, but that might be a problem in some environments - inclusivity. Our industry tends to be hostile towards people that do not match the image of a typical programmer, be it regarding their gender, age, but also their education - CS graduates might feel superior to other programmers and it is our job to ensure that everyone feels welcome in the team, that everyone feels valued. Except for assholes, assholes should not feel welcome.

Different paths to success

Coding bootcamps are here to stay, whether we like or not. With the current trends like growing demand for software developers, wider access to informal education, with universities struggling to deliver good quality of lectures while they are forced to teach remotely, we are going to see more and more people taking alternative ways to become programmers. I personally believe that it is great, I am excited about the democratization of this process. We've seen it with MOOCs, we see it with bootcamps, we'll see it with other ways. It is wonderful that the pool of talent is not limited by their ability to spend 3-5 years studying CS. As managers, as leaders, as experienced developers we should treat it as a challenge and opportunity, not a problem.

Credits

"For hire" photo by Clem Onojeghuo on Unsplash
Pottery photo by SwapnIl Dwivedi on Unsplash

An idea on how to improve software versioning and  managing dependencies came to my mind a few days ago. I'm yet to figure out how it would exactly work, but I thought I'd share it first.

Who follows Semantic Versioning?

Semantic Versioning is a great idea, a lot of projects aim to follow it, and yet not all of them do it in 100%. The main issue is increasing major version whenever a backward-incompatible change is introduced. People don't want to release MyLib 2.0 when the current version is 1.3 and the new release fixes just 1 bug in a way that in some cases breaks backwards compatibility. On the other hand as someone whose application or library depends on MyLib, I'd like to be able to safely update it across all 1.x versions without worrying that some change will break my code.

And when some project really increases major version even for tiniest incompatibility, I find is scary to see that 3 months after upgrading library from v5.0 to v8.0, now I need to update it to v12.0, because there were a few minor, but breaking changes. Normally I expect major version to introduce, well, major changes - and while I know that this is not what SemVer claims, I can't stop thinking that moving from v3 to v4 should bring something more than a single security fix.

I believe my expectations are not really baseless - for most of larger libraries I've used, the next major version is something maintainers plan carefully, consider breaking changes, introduce new features and sometimes do quite massive rewrites. They keep using X.Y.Z versioning and refer to releases as major, minor and tiny. Yet they don't follow Semantic Versioning strictly.

SemVer -> breaking.json?

So I've been thinking what can be done about it and I thought - what if we can mark breaking changes in the repository in another way? Let's say by having one file like breaking.json that would contain list of versions with breaking changes:

{
  "3.5.0": [
  	{
      "change": "myProperty function has been removed",
      "details": "https://github.com/MyName/MyRepo/issue/4321",
      "impact": "low"
    }
  ],
  "3.0.0": [
    {
      "change": "myname function renamed to myName",
      "details": "https://github.com/MyName/MyRepo/issue/4002",
      "impact": "high"
    }
  ]
}

This way when running npm update or some other command I could see what are the breaking changes happening between my current version and the version to which I want to upgrade, with some details and potentially instructions what to do, and then I could decide whether I want to proceed or pick lower version instead. It could be run with some predefine settings - always stop when incompatibility found, or always ask what to do, or print the list of the latest versions I can use without any backward-incompatible changes. There are many possible ways to develop this idea and while it would make updating libraries a bit more involving, it would give developers information about what changes right on their screen.

Pros

• minor breaking changes could be done in minor releases - potentially authors could be introducing small breaking changes in a few minor releases instead of dropping massive upgrade guide once every 2 years
• allows to keep major versions for major changes
• allows user to find out what is the latest version without breaking changes ant update to that particular version
• this is a bit far fetched: it could potentially allow to override strict dependencies - let's say MyApp uses MyLib 1.5 and MyOtherLib 1.3. I want to update MyOtherLib to 2.0, but I can't, because MyLib depends on MyOtherLib version 1.x; currently I need to ask maintainer of MyLib to bump dependency or fork the library. With this feature I could see myself list of breaking changes and if I feel it's safe, I could force installation of MyOtherLib 2.0 instead.

Cons

• requires additional support from the package managers: NPM/Bundler/Hex and other package managers would need to approve such feature and maintain the support
• the dependency management would become more tricky, let's say I'm updating my dependencies and one of my indirect dependencies has a breaking change - what should I do? I don't even know the library, how can I decide whether it's safe to update it or not?
• requires maintainers to keep a separate list of changes in another file and to keep it up-to-date (unless it becomes part of the changelog?)

Is almost-SemVer the best we can get?

My idea is rather idealistic and probably quite challenging to implement. I'm also not sure whether it would actually improve the situation. Maybe the current way where people mostly follow SemVer is the best we can get and having some breaking changes here and there is the cost of relying on other software?

I'm not sure whether there is any practical way to improve the situation, but I feel adding list of breaking changes that could be displayed to the user while upgrading is at least a step towards more awareness and it could save open source maintainers some time they spend answering the same questions and closing duplicate issues.

Do you think there are any other ways to improve software versioning and managing dependencies?

Leave your comments here: https://news.ycombinator.com/item?id=24571722

Recently I've been reflecting on the (loosely defined) quality and popularity of programing languages. Despite growing appreciation for static typing and compilers (whether in fully-compiled languages like Java or with JIT recently introduced in PHP or Ruby), 2 dynamic, scripting languages pushed their way up in the popularity rankings in the last couple of years. These languages are JavaScript and Python.

While JavaScript has had a huge advantages of having its interpreter installed on most of computers and of being the only language that could be directly executed in the browser, Python's success on the surface seems more unexpected and less obvious. In this post I'm going to think and try to find out why Python has won against other, similar languages.

If you can think some other reasons you'd like to share, drop me a message or leave a comment in the discussions linked below. And if you enjoy my content, make sure to check out my YouTube channel where I share career-related advice for programmers.

Competitors

In a relatively short period of time, between 1987 and 1995, a few major and influential languages were created. Among them there were a couple of high-level scripting languages that gained a lot of popularity and that remain used and are developed until today:

• Python was created by Guido van Rossum, a Dutch programmer working at a research institute. Inspired by ABC, created at the institution where van Rossum worked at that time, Python was initially created in late 1980s. Driven by values like explicitness, simplicity and readability, 30 years later Python is among the most popular languages in the world and is heavily used in many areas like machine learning, academia or web development
• PHP wasn't really meant to be a language, but rather a preprocessor which was supposed to help generating HTML more dynamically. Created in 1994 in Canada, PHP quickly gained popularity as a new, easy way to create websites that can connect to database and dynamically generate content before sending it to the user. It's been 20 years since arguably the most influential version of PHP was released (4.0). Today PHP team aims to release version 8 soon, and the language they develop, despite a lot of competition, continues to power a significant portion of websites and web applications all around the world.
• Perl might not be so well known by younger readers due to a complicated history over the last 15 years, but once it was a language of hackers and programming enthusiasts. Its development started in 1987, the last major version, Perl 5,  was released in 2000, but a new one, Perl 7 (the history of Perl 6 is complicated) is just around the corner. Perl does not have a single purpose like PHP. While it has been used extensively as a backend server language, it's also used for some system tools in utilities in GNU/Linux.
• Ruby is a creation of Japanese programmer, Yukihiro Matsumoto, who wanted to create a language that would be pleasant to work with. Programmer's happiness, freedom and flexibility are values that drove the development of the language. The project waas started in 1993, and in December 2020, a new major version, Ruby 3, is expected. While Ruby gained worldwide popularity thanks to a web framework, Ruby on Rails, and is often considered a web programming language, it has significant usage in other domains like server tooling and, primarily in Japan, embedded devices
• JavaScript is kind of a guest in this article, as I can't say that it's clearly less popular than Python, so I will mention it here and there. Created in 1995 in USA it started as a language to be included in Netscape Navigator browser. Its goal was to add more dynamic elements to websites. It later became de facto standard for other browsers, and in early 2010s, with creation of NodeJS, it moved out of the browser to become a general purpose language. While it remains primarily used for the web, we see an increase in number of desktop applications and various utilities created in JavaScript (or one of languages compiling to JS like TypeScript).

What made Python #1

Features?

When I think about the languages above, I can't clearly choose the best one in terms of capabilities. While Ruby has been my go to language for many years, I understand why others might prefer basically any other language from the list and they can make very valid arguments to justify their choice.

Over time all these languages became more and more similar to each other. Don't get me wrong, there are still major differences between them - different philosophies, syntaxes, different legacy issues - in the recent years they all became much closer to each other in terms of features though. So while I am sure that certain features can drive the initial interest and usage of the language (PHP made it simple to generate dynamic websites, Perl's syntax was created with linguistic principles in mind), I don't think that Python won because of having tuples, shorter syntax or enforced indentation. There must be more than that.

Corporate adoption

A strong corporate player which adopts some language in its own software and openly declares it can be a huge boost for the language popularity. Even more if the language is created by the company. The success of Java as an enterprise language is owed at least in part to marketing efforts of Sun. C# would never reach its current position if it wasn't created and promoted by Microsoft.

While Python was created as a hobby project and not a business tool, it got some strong support in 2000s, with Google as the biggest company adopting Python. Neither of the other languages has ever received such a strong support. Even though Facebook was initially written in PHP, today it's hard to see any PHP code among their open source projects (beside Hack, their language built from PHP).

Corporate support does not guarantee success - Dart created by Google is still actively developed, but never reached popularity of Python or even Go (another Google's language). Therefore it can't be the solely reason explaining Python's popularity over other languages, but it gave Python a big boost in 2010s and the age of data science.

Academia and first mover advantage

I'm pretty sure you know the history of QWERTY - the keyboard layout used by nearly 100% of people using Latin alphabet (with some minor modifications in various countries). QWERTY was not designed for fast typing - quite the opposite, its goal was to prevent people from typing too fast, because that blocked the typing machines, which were very common before invention of modern computers. The old limitations do not apply anymore, yet we still use the same suboptimal layout. Why? Because everyone else uses is. Because almost every keyboard in the world is designed with this layout in mind, and making people switch would require enormous effort.

It's quite hard to change habit as an individual, and even harder as a large group of individuals with their own opinions. I believe this is one of the reasons behind lasting popularity of some programing languages. Let's take PHP - since early 2000s it's been supported by basically any hosting provider. If you wanted to run applications in Python you had to buy VPS and install the interpreter on your own, or ask the admin to provide it for you. And PHP was already there, so why not to use it instead?

This might be a bit of a stretch, but I think a similar thing can be said about Python in academia. Yes, technically it wasn't the first language used by academic institutions, but I believe it was the first language that allowed scientists to write high-level code and provided them with large set of functions that allowed scientists to work with complex numbers and equations. While significantly slower than PHP, Python allowed everyone to quickly build prototypes which later could be moved to C or C++ if performance was the problem.

With the rise of open source and growing number of available libraries, Python became go-to scripting language in academia and while there are other tools that might be more popular (looking at MATLAB for calculations and Java as intro to programming for students), Python's position is very strong, and any language that will try to take its position has to compete against 20+ years of work on libraries and tools written for scientists.

The rise of the new

Together with the two reasons above comes timing and the changes happening in software industry. In the last few years the large web frameworks started losing steam. Growing popularity of microservices and function-as-a-service, together with some benefits of using the same language for the client and the API, pushed adoption of JavaScript as the language of the web (there were more reasons for sure). Other languages which were primarily used in the webdev had to either find other niches or accept losing popularity.

At the same time another area started gaining huge popularity - data science. Such a powerful, new discipline had a chance to open door for new players - in terms of business, technological advancement, and (among many others) programming tools. It also helped Python to establish and strengthen its position as the programming language on the verge of science and engineering.

1+2+3 = Python the King

Let's add these 3 points together. It's some time between 2010-2015. Python together with R and MATLAB are the programming environments used by scientists. At the same time big tech companies know that machine learning is going to grow quickly and they build tools to enable scientists and engineers to use the potential of modern computational power combined with neural networks. Google works on TensorFlow, which will become the most popular machine learning framework. What do they use? The language of the science, the language they've adopted a few years before - Python. Of course the implementation of TensorFlow is done in CUDA and C++ due to performance, but the programming interface for its users is provided in Python.

The rest seems rather straightforward from then on: Facebook's PyTorch, a competing tool, also leverages Python. Data Science bootcamps - which appeared so suddenly and in such a number like mosquitos at the lake in a hot, sunny summer evening - teach Python. Your neighbor creating Udemy courses for beginners also teaches Python.

Since Python has existed for more than 20 years and has been used in various industries and applications, its growth thanks to data science pushed forward adoption in other areas - Django and Flask get users who came to Python for data science, but stayed for good.

Why not other languages?

Python's story seems so easy and obvious in hindsight, but there were a lot of obstacles on the way. The infamous migration from version 2 to version 3 could have been (and I guess for many people was) a disaster. Till today creators of programming languages say "we don't want to repeat Python's scenario" when explaining their decisions regarding breaking backwards compatibility. Python's packaging tools (famous "- how to install pip? first install easy_install", moving from eggs to wheels as packaging format) were behind those available in other languages. Eventually it didn't matter though, despite all these issues, Python not only maintained its position, but even gained a lot of popularity over the years.

Finally let's have a look at the other scripting languages that I have mentioned in the beginning and think and wonder about why it wasn't them that reached such popularity in recent years:

PHP

PHP actually might have reached similar level of popularity around 2002-2008, and it is still going strong when it comes to web development. However it's never been a general purpose language. While it is possible to write a PHP script that will act as a Unix system tool, the language at its core is meant to be used for server side of web applications.

On top of that, I believe PHP has certain characteristics that make it more difficult to use by larger teams (not saying it's impossible, just more difficult) - weak typing, lack of arrays (its number-indexed arrays are actually dictionaries), inconsistency in many places like function names or order of parameters - I guess one of the reasons why Facebook created Hack was to get rid of a lot of legacy stuff in PHP.

JavaScript

As I wrote in the introduction, I believe JS might be equally or more popular than Python, but despite its quickly growing popularity in early-to-mid 2010s it is not clearly the number 1 dynamic language. Why hasn't it dominated the entire space occupied by dynamic languages after it dominated the web? I'd say the first reason is that until a few years ago JS wasn't even a good programming language. I spent a couple of years using CoffeeScript instead, simply because it was so much more pleasant to write. It changed with the ES6 and new features introduced to the language, but that was quite late. JS now and JS in 2012 are 2 very different worlds, and years ago when NodeJS was introduced there was a strong resistance to it. A lot of programmers didn't want JS to spread to the backend where there were many other options to choose from. On top of that, JavaScript for years was considered rather a toy, a language to add animation on the website, not something to use for large projects, so it took time to convince people (and especially backend programers) that it's stable, mature and good enough for more advanced tasks.

Perl

I've never done any serious work in Perl, so I don't really know why it failed to keep growing in popularity. Based on my understanding of the language and its development (as well a few hours spent on reading about this topic), Perl in its current form as of Perl 5 reached its limits in terms of attractiveness to programmers. It was great for scripting, but it was too hard to maintain large codebases in it. And the development of Perl 6, a version that was supposed to fix these problems and make language clearer, took 16 years to be released and eventually in 2019 the language was renamed to Raku (and is not considered a successor of Perl 5). By then a large number of developers have already moved to other languages like Python or Ruby. Interestingly, Python also had a rocky migration, but it was eventually successful, unlike moving from Perl 5 to Perl 6.

Ruby

Since I'm personally a fan of Ruby I might be biased here, please keep that in mind.

To me Ruby and Python are very similar languages. Their philosophy is different, syntax is different, but Ruby programmer reading Python code is like a Spaniard visiting Italy - you don't understand everything, but it's enough to make it work. So I think why one became more popular than the other must have reasons beyond the language itself.

The first reason that comes to my mind is its origin - Ruby was created in Japan and it wasn't widely known in Europe and America until 2000-2004 when first a book about Ruby was written in English and then Ruby on Rails framework was created. By then Python was already gaining popularity in academia, while Ruby was categorized as web development language in the western world. What's more, the development of Ruby used to be often discussed in Japanese, which is perfectly ok and understandable (since most of contributors were/are from Japan), and at the same time I feel it might have impacted the worldwide language adoption.

The 2nd thing is the categorization that I mentioned above - while Ruby is a general purpose language, besides Japan it is primarily known because of Rails framework (and maybe Chef and Puppet that had their share of popularity). So while it became very popular in the age of monolith SaaS applications, when the web dev moved more to JS, there was no other niche where Ruby dominated.

There are a few more potential reasons that come to my mind, but that's for another time.

Long Live the King - but how long?

Python's dominance among dynamic languages and its huge growth of popularity is a fact. The question remains: how long will it last? Or rather, what next major shift in programming will change the situation again? Maybe the next generation of low-code tools? What languages will we use then? Will they be even closer to English than the languages we have right now?

I have a lot of questions and few answers. I know that with the growth of computational power of our devices, with the growth of popularity of machine learning and AI, and finally with growing need for tools that will let everyone control and program the devices we use daily, we must be and we are moving towards tools of higher level. While there will certainly be space for more languages like Go or Rust in the future, they're tools for experts, not for the masses. Python and other current high level languages feel like kind of a bridge - they can be learned faster, they look similar to English, yet they still require a lot effort to build non-trivial tools. I predict that the next languages that will take over Python and JavaScript in popularity will be vastly different and will allow us to express ourselves in a more natural, human way. Who knows, maybe they will even be implemented in Python.

Discussion

Since I do not maintain comments feature on this site, please leave a comment on HN: https://news.ycombinator.com/item?id=24529615, I'll be following that thread

Sources

• https://www.python.org/doc/essays/foreword/
• https://stackoverflow.com/questions/2560310/heavy-usage-of-python-at-google/2561008#2561008
• https://www.python.org/doc/essays/foreword/
• https://docs.raku.org/language/faq
• https://wiki.php.net/rfc/jit
• https://blog.heroku.com/ruby-just-in-time-compilation
• https://opensource.facebook.com/projects
• https://github.com/tensorflow/tensorflow
• https://github.com/pytorch/pytorch
• https://news.ycombinator.com/item?id=4495101
• http://web.archive.org/web/20130118230744/http://dev.hasenj.org/post/31042963934/nodejs-hate
• https://www.fastcompany.com/3026446/the-fall-of-perl-the-webs-most-promising-language
• https://speakerdeck.com/stevan_little/perl-is-not-dead-it-is-a-dead-end
• https://en.wikipedia.org/wiki/Hack_(programming_language)
• https://engineering.fb.com/developer-tools/hack-a-new-programming-language-for-hhvm/
• https://thehftguy.com/2019/10/07/perl-is-dying-quick-could-be-extinct-by-2023/
• https://www.forbes.com/sites/forbes-personal-shopper/2020/09/11/best-tweezers-for-hair-removal-splinters/#5d0e1e17198a
• https://trends.google.com/trends/explore?date=2010-08-19 2020-09-19&geo=US&q=%2Fm%2F0jt3_q3,%2Fm%2F01hyh_
• https://www.calu.edu/academics/undergraduate/bachelors/data-science/jobs-career-salaries.aspx
• https://www.zdnet.com/article/data-science-dominates-linkedins-emerging-jobs-ranking/
• https://www.quora.com/Why-is-Perl-no-longer-a-popular-programming-language
• https://packaging.python.org/discussions/wheel-vs-egg/
• https://jakevdp.github.io/blog/2013/01/03/will-scientists-ever-move-to-python-3/
• https://stackoverflow.blog/2019/11/14/why-is-the-migration-to-python-3-taking-so-long/
• https://lwn.net/Articles/758159/

Credits

Photo of Larry Wall - By Randal Schwartz from Portland, OR, USA - Flickr, CC BY-SA 2.0, https://commons.wikimedia.org/w/index.php?curid=2938351

Photo of Yukihoro Matsumoto - By Cep21 - Yukihiro Matsumoto. Originally uploaded by Cep21 to English Wikipedia., Public Domain, https://commons.wikimedia.org/w/index.php?curid=2706367

Photo of Brendan Eich - By Darcy Padilla - https://web.archive.org/web/20140209081556/http://blog.mozilla.org/press/bios/brendan-eich/ https://web.archive.org/web/20131108073412/https://blog.mozilla.org/press/files/2012/04/Thumbnail-Full_Eich_04.jpg, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=31783773

Photo of Guido van Rossum - By Photograph by Daniel Stroud, first retouched version uploaded by User:Deedub1983, second retouching by User:HarJIT. - Retouching (dynamic range compression) of File:Guido-portrait-2014-curvves.jpg, itself an exposure reduction of File:Guido-portrait-2014.jpg, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=82866015

Photo of Rasmus Lerdorf - By William Stadtwald Demchick - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=34989289

Flask logo - By Armin Ronacher - http://flask.pocoo.org/static/logo/flask.svg, Copyrighted free use, https://commons.wikimedia.org/w/index.php?curid=19501815

Django logo - https://www.djangoproject.com/community/logos/

Numpy logo - https://commons.wikimedia.org/wiki/File:NumPy_logo.svg#/media/File:NumPy_logo.svg

Pandas logo - By Marc Garcia - https://github.com/pandas-dev/pandas/blob/master/web/pandas/static/img/pandas.svg, BSD, https://commons.wikimedia.org/w/index.php?curid=73107397

Tensorflow logo - By FlorianCassayre - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=58380451

Pytorch logo - By https://github.com/soumith - https://github.com/pytorch/pytorch/blob/master/docs/source/_static/img/pytorch-logo-dark.png, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=71168078

Ansible logo - By Ansible.com - https://github.com/gilbarbara/logos/blob/master/logos/ansible.svg, Public Domain, https://commons.wikimedia.org/w/index.php?curid=52280850

SciPy logo - https://www.fullstackpython.com/scipy-numpy.html

Photo of Python - Photo by Divide By Zero on Unsplash

Google logo - https://www.google.com

University hat icon - By Leon Rische https://thenounproject.com/l3kn - https://thenounproject.com/search/?q=university hat&i=463808, CC BY 3.0, https://commons.wikimedia.org/w/index.php?curid=57981381

Ruby logo - http://ruby-lang.org

The Camel of Perl - https://www.perl.org/