Corgibytes

Secrets to Managing a Software Modernization Project

Tue, 13 Dec 2022 00:00:00 +0000

“How in the heck can you love working in software maintenance and modernization?” This is a question I’ve gotten asked quite a lot over the years. To many people, working on challenging, older, complex, or brittle systems is something they like to run away from. But not us. There’s a lot of satisfaction in improving the software that is often at the heart of our client’s operations. Among other reasons, it’s meaningful to work on projects that matter.

However, a critical component of success also has to do with the way we run projects. Over the years, we’ve compiled several tenets to our approach that help make a project run smoothly. These seem like secrets because many of them are just so different from what you find in traditional operations, but these ideas aren’t new. Many of them are inspired by the Agile Manifesto, particularly its twelve principles. The difference is that instead of looking at Agile as a methodology and debating the merits of sprints and story points, we’ve always taken it more as a guiding philosophy that we can use to build our own way of working.

As someone with a background in business, the first time we experimented with some of these, I admit that I bristled. I loved the feeling of certainty, perfection, and accuracy. When members of the team mentioned these as project requirements, I was often confused because it was just so different from what I was familiar with. A decade in, however, I’ve learned that, at least when it comes to software modernization, these practices work.

Focus on the System, Not Just the Components

Your health depends on more than just the biological mass of your body, and your software is more than what’s in your source code. At Corgibytes, we take a holistic approach to code health. In addition to helping you pay down technical debt and add new features, we also look outside the codebase to help discover and resolve root causes that may not be immediately obvious. Some examples include: DevOps, Architecture, Documentation, Organizational Structure and Culture, Team Dynamics, Business Initiatives, and more.

Embed Empathy into the Process

Empathy is an essential skill for healthcare workers and software developers. For over a decade, we’ve woven our core value “Act with Empathy” into our operations and daily development practices. When empathy is applied directly, we have seen stunning positive results. For the past two years, I’ve been digging even deeper into this principle and am currently working with Pearson to publish our learnings into a book. Keep an eye out for Empathy-Driven Software Development coming in 2023.

Communicate for Historical Mutual Understanding

All code is communication. We don’t code for the compiler, we code for people. The ability to make progress on a codebase largely depends on how well you understand the system and the context that you’re developing for. This means working with people who have different skills and experiences and finding ways to proceed with joint solutions. Additionally, people on a team come and go. As we build our understanding, we emphasize creating durable artifacts of our work so that the people who come after us have an easier time understanding why choices were made.

Maintain Technical Excellence

Honing your skills requires a commitment to professional development. Early in my career, I worked at a company where I was expected to be 100% billable and then somehow squeeze in professional development. But you can’t add to something that’s already full. So when we had the chance to build professional development into our operations, we did. While it can be awkward to have conversations with clients about why we won’t staff someone on their project for more than 36 hours per week, this small reduction in billable hours delivers lots of value to a project by helping to ensure our team members are rested, focused, and building new skills.

Be Prepared for Surprise and Change

Even with effective planning and preparation, things don’t always go the way we expect. This is why our operations lean heavily on practices that allow us to be responsive. We don’t adhere to a specific type of methodological dogma to achieve this. To us, Agile is much more than sprints, standups, and story points. While team rituals can be incredibly beneficial, rote execution of a predefined process often fails. We focus on pragmatism, excellent communication, collaboration, trust, and empathy. We’re not afraid to experiment with new ideas and enjoy co-creating processes that work for your situation.

Make Safety a Prerequisite

We admit it — we took the wording of this principle directly from the Modern Agile movement. In many cases, we recommend activities that are designed to stabilize the system and mitigate risk. This is rooted in a sense of safety, protection, and ethics. Some clients have found it frustrating when Corgibytes advocates strongly that significant safety concerns are prioritized over deploying new features. For example, making sure the code lives in a source control system instead of in a local environment or that customers’ passwords need to be encrypted. These recommendations come from a sense of care, duty, and recognition that having a strong foundation is the key to long-term health.

Estimate Is a Verb, Not a Noun

Responsible fiscal management is a critical aspect of any project. Budgets are real and it’s important for businesses to manage their expenses. That’s why we approach estimating as an ongoing activity instead of a static deliverable. The software systems we are specialists in working with are inherently complex with significant uncertainty, like the weather. This means that linear project management often fails, wasting precious time and budget. Our approach to estimating is to continuously ask the right questions, communicate using ranges and confidence values, and regularly refine the definition of done that will help you meet your overall business and project goals.

Build Hygiene Habits

Health, whether that’s in humans or software systems, is an ongoing activity. Just like you need to brush your teeth every day to prevent big problems later, there are small development practices that when performed regularly make a big difference. This includes activities such as upgrading languages, frameworks, and third-party dependencies, writing code in a way that’s intention-revealing, taking a test-first approach to writing and refactoring code, extracting methods so they don’t become too complex, resisting the copy and paste urge to avoid duplication, and many, many others.

Optimize for Resilience and Longevity

Too much short-term thinking can lead to brittle systems that are prone to sudden breakage. That’s why we constantly keep our eyes on the horizon, balancing short-term needs with long-term goals. When a codebase is healthy, it can recover quickly when it becomes stressed and operate sustainably into the future.

Whole Team Approach

Healthcare teams are much more than just a doctor. Behind the scenes, there are teams of nurses, technicians, administrators, janitorial staff, assistants, and specialists who all work together in concert to give you the care you need to improve your health. This is why Corgibytes works with a single blended rate. While you may interact with individual Code Whisperers on a regular basis, there is a whole team behind them to support their work.

Gradual Gradients over Brusque Binaries

Effective change often happens gradually over time. We’ve seen this time and again when it comes to digital transformations. Hard cutoffs can be tempting but they rarely go well. Our approach is more diligent. By replacing a system over time, eventually, you can get a brand new app with less risk and the benefit of operational continuity. When you think about it, this is exactly how our bodies work. Cells are constantly regenerating at different rates and every few years, you might even consider yourself a new person.

Business Goals Drive Recommendations

Just like a doctor’s treatment plan will vary based on their patient’s goals, our recommendations are not one-size-fits-all. The better we understand your business’s broader strategy, the better we can devise solutions that will help you achieve them. This way, we are constantly working together to assess your Return on Investment and can adapt as needs change.

Software projects work best when we don’t just talk about adaptability and communication, but actively incorporate them into our operations. Learning to love legacy code has been a joyful journey for me, and I hope it will be for you, too.

Spice Up Legacy Code Leftovers

Tue, 15 Nov 2022 00:00:00 +0000

What to consume tonight? You dig around your server and don’t find much. You think maybe it’s time to create something new or buy a pre-made 3rd party application. But wait… at the back of the server you find an old application. A compiled application that you really like and is still consumable but does not taste as good as it used to. It is also starting to smell a bit funky, like an old pair of legacy socks.

You could consume the application as is but would really like to freshen it up so it tastes better. You are also worried that consuming it as it might upset your delicate system and cause you to throw an exception or worse, a massive core dump.

You could create a more palatable version if you could find the source code. You dig around the server and success! Tucked in the bottom rack you find the source code just sitting in a folder. The problem is that is all you find. No documentation. No source code control. Nothing else. Just the application running in production and the source code.

Putting the source code on the counter you walk over to your shelf with all the cookbooks. Where is it… that old recipe book your grandparents gave you. Ah-ha! You pull out the Legacy Code Cookbook. This should have the recipe you need.

Legacy Code Leftover Casserole

A recipe to freshen up an old compiled application. Assumes you can only find the source code, database, and nothing else.

Ingredients

Compiled application with database running in production.
Source code for the application.

Tools you will need

Version control (e.g. Git, GitHub, Mercurial, BitBucket, GitLab, SVN, etc)
IDE appropriate for the application. (e.g. VS code, Intellij, etc)
Local or remote test DB that matches production. (e.g. if production is running Postgres 8 you should run the same locally or in test)
Containers (not required but makes life easier). (e.g. Docker)
Staging environment that matches production as closely as possible. (e.g. if the production environment server is running Ubuntu 16.04 so should staging)

Steps

Add the source code to version control including any assets (e.g. images) used by the application. If available also add database files such as schemas and/or migrations. While most files should be added to version control some break this rule such as environment config files (e.g. files with passwords and environment specific values), log files, and files generated by the application (e.g. report files).
Create a backup of the production database. This is also a good time to set up scheduled production backups and a data retention plan. The frequency of the backups will depend on how costly losing data is. For example, if losing a day’s worth of data will not have a significant impact on your business then daily backups are fine. The data retention plan should follow local laws and regulations for your given country and/or industry.
Examine The existing code at a high level and look for any indication that source code you have does not match production. For example, the application in production has a screen or report that you don’t notice source code for. If you find a major discrepancies then you should try to find a more recent copy of the source code. If that does not exist then you need to use a different recipe then this one as the rest of this recipe assumes you don’t find any major discrepancies.
Restore a backup that is accessible to your development environment, which will be setup in the next step, but is also secure as the backup contains production data. If you want to restore the database on your local laptop, or other in-secure location, the data should be anonymized first. Where you restore the database should also follow any local privacy laws and standards.
Get the source code to compile and run. This will likely be the longest and hardest step as you figure out all the dependencies the application needs. The easiest way to do this is to create a development environment in a container (e.g. Docker container) or virtual machine that matches your production environment as closely as possible. Any config settings will need to be updated for your development environment. Hopefully they aren’t hard coded in the application, if they are then you will have to extract them to .env or similar file.
Run the application and see if anything obvious is broken. Specifically focus on the critical paths of the application. For example, if it’s an e-commerce application can customers make purchases?
Get the automated tests to run and pass, assuming there are any automated tests. Worst case scenario you can ignore failing tests but only as a last resort.
Run the application and this time explore the entire application and make sure everything works. If possible, compare it to the production version. Fix any issues you find and also add some tests. You might have to mock out or set up development accounts for 3rd party services.
Create a staging environment and, if possible, use infrastructure as a code tool (e.g. Terraform, Ansible, etc) to create the staging environment.
Install the application in staging. A good time to document and/or automate the install process.
Test the application on the staging environment. Ideally you have non-developers doing some testing at this stage, if possible.
Switch the staging environment to production. How you do this will depend on your application. Monitor for issues and address them as they arise. You likely missed something but now you are in a good position to make fixes, do a staging release for testing, and then push to production.

Now that you can recreate the original application from scratch you can modify as needed. Freshen it up. Remove dead parts that smell bad. Enhance it with new flavors. Basically make the application more palatable for yourself, and others that consume it.

Creating Secure Smart Contracts for the Blockchain

Mon, 31 Oct 2022 00:00:00 +0000

As a software company that specializes in modernization projects, helping teams guard against security vulnerabilities is a big part of our work. Most people think this work only applies to large legacy systems that are running Java Enterprise or .NET. While we definitely work on those projects, security vulnerabilities can creep in on even the newest of projects.

Security vulnerabilities are something you want to avoid in pretty much any industry, but when it comes to people’s money and sensitive information, the stakes are even higher. Given all the buzz around Decentralized Finance (DeFi), I was curious — how are teams in this space doing when it comes to managing their security vulnerabilities?

Solidity, the most common programming language for writing Smart Contracts (computer programs that are hosted and executed on a blockchain network) on Ethereum and Ethereum Compatible (EVM) blockchains, is a powerful language. However, there is a caveat, writing secure Smart Contracts without vulnerabilities is much harder than just learning the correct syntax and getting the contract to compile and execute. And learning lessons of the vulnerabilities that can arise in a contract has been very expensive, into the millions and even billions of dollars. There is a lot of great work being done to help mitigate the vulnerabilities and the risk of funds. However, in almost all cases, especially in recent months, the loss very likely could have been avoided by enforcing good software development practices, most notably code coverage rules and requiring code reviews.

A Bit of Background About Smart Contracts

Like many people, I had been hearing the murmurings of Bitcoin, The Blockchain, Smart Contracts and Crypto Currencies over the past several years. At first, I mostly dismissed it as another technology trend that sounded interesting and probably had a lot of merit, but nothing that I should let distract me.

However, toward the end of 2021 The Blockchain started becoming more and more interesting to me. I began digging into it little by little, gaining a deeper understanding of the core Blockchain technology, Smart Contracts, Mining, and DeFi. As a software developer who has spent many years creating, maintaining and upgrading new and old software, the immutability of a Smart Contract really struck me. In an enterprise development environment, the goal is to release quickly and often. The idea of not being able to update or modify the contract once it has been deployed was a bit mind blowing, and something that took some time to really comprehend.

This made me wonder how someone could write an immutable Smart Contract. I would assume with an overabundance of tests and rigorous code reviewing. Otherwise, how would you know that it is doing what it is supposed to do and not doing the things it is not supposed to do? Additionally you have to know this with certainty before the contract is deployed.

Along the way, I began learning about what promises to be a useful subset of Smart Contract projects, which is commonly known as DeFi. DeFi is an umbrella term for peer-to-peer financial services on public blockchains. This is a very interesting use case for Smart Contracts and The Blockchain as it has a tremendous amount of potential for many types of people. However, not long after exploring this ecosystem it became apparent that my theory about immutable Smart Contracts being well tested and reviewed before deployment was not true…

The Costs of Vulnerability Mismanagement

In February of 2022 the DeFi project Titano Finance lost $1.9M due to an exploit left by a contract developer. In April of 2022 another DeFi project called Elephant.Money lost $11.2M due to a small coding oversight in their Smart Contract. In a matter of weeks and months, DeFi project after DeFi project experienced some sort of loss in funds and reputation due to exploits in vulnerable code.

Vulnerabilities were created either intentionally by malicious developers or mistakenly by well intentioned developers. Some projects survived while others did not. Millions and millions of dollars were stolen by bad actors. Many of these bad actors likely did not even have crazy hacker skills one might expect for the amount of money stolen. Some of these exploits simply took advantage of sloppy coding and development practices while others were due to these projects having untrustworthy contractors or employees who created the vulnerable code intentionally.

The publicity this draws can lead more and more people to the idea that The Blockchain, more specifically, DeFi, is the Wild, Wild West all over again: no enforceable laws, malicious actors, and a generally dangerous and unpredictable landscape. This can lead many people to shy away from crypto currencies, The Blockchain and DeFi specifically. But is it really the Wild, Wild West again? And if so, does it have to remain that way? Let’s explore a bit to understand possible ways to solve the problems.

Taming the Wild West

But first, a caveat. Let me be up front and tell you that I am not a historian. Almost all of my knowledge and understanding about the Wild, Wild West comes from Hollywood’s portrayal of the era, so it’s important to acknowledge that we are leaning on tropes here. The metaphor is still useful to communicate the ideas I wanted to explore in this post, so let’s go with it for now.

Back to the analogy: The West that was once considered so Wild and lawless long ago is now considered some of the best land with some of the most desirable places to live in the country. How? Well, sure some of it was due to rules and regulations of governments and their enforcement, in two words: The Law. But, another contributing factor was good people with the right tools and abilities that desired to help tame the Wildness of “The West”. Some people, farms, towns and cities hired these people of good character and merit to help protect them from the dangers and maliciousness of that time. These benevolent characters brought peace and security to the areas under their responsibility.

Well, this same idea is also becoming part of the landscape in DeFi specifically and The Blockchain as a whole. There are good people with the right skills, abilities and character who are helping tame this new wild, wild west and trying to make it safe for all. One such developer was paid $6M by Aurora Labs for saving 70,000 ETH (worth about $210M at the time the bug was reported). Another developer was paid $10M for finding and fixing a bug in Wormhole bridge. Projects like Code4rena and Immunefi are helping match DeFi projects with developers who have the skills to help find and resolve vulnerabilities before funds become at risk. Organizations like OpenZeppelin, ConsenSys and Gnosis are helping developers build better and more secure Smart Contracts in the first place with helpful tools and libraries.

Where Do We Go From Here?

Even though DeFi might feel new and shiny, it’s still susceptible to many of the same problems of any other software system. As the DeFi industry matures, here are some ideas for teams who are looking to scale safely.

Balance Optimism with Diligence

When we’re overly optimistic, we can inadvertently ignore important problems. Over the years, we’ve found diligence to be an excellent counterbalance to optimism bias. Adopting a pragmatic stance helps us spot problems early, when they’re easier, cheaper, and safer to resolve. As mentioned earlier, in a number of the cases, diligent code review practices quite possibly could have caught vulnerabilities before they had been released to the blockchain and put funds at risk.

Cover Critical Code With Tests

Think of your automated test suite as a safety net. All of your critically important logic, especially anything that has to do with financial transactions, should be covered by tests. Unfortunately, one of the big challenges in building Smart Contract tools such as solidity-coverage, which can help you determine which logic is and isn’t covered, are still in their infancy. If you want help getting started, overcoming inertia, or maintaining momentum on these efforts, Corgibytes is happy to help.

Code with Compassion

If you’re writing code in the DeFi space, remember the importance of empathy, which is at the core of the Corgibytes culture. At the end of the day, what you’re building has a real impact on real people. Don’t lose sight of the humanity in your work.

Watch Out for Hero Culture

As the industry moves away from its Wild West roots, it’s tempting to hail the heroes who come in and save the day. However, celebrating heroes can backfire. It creates an environment where people are incentivized to create and hide problems.

Calm the Chaos

New ideas are exciting, and sometimes a bit of short-term stress can help a project cross a major milestone. But when a culture of urgency becomes the norm, it’s easy for even the most diligent of developers to make mistakes.

Remember that Trust Isn’t Absent

While the big appeal of DeFi is the chance to make profits off of what promises to become a digital alternative to Wall Street, there is still a need for Trust. One of the promises of blockchain technology is the idea of Trustlessness. Without going into a lot of detail about the merits and drawbacks of Trustlessness, the basic premise is that the parties involved in a transaction do not have to trust each other as long as they can trust the contract. But therein lays the challenge with DeFi. The DeFi contracts that have come on the market have rather complex logic, making it difficult for the average investor to read and understand them. Trusting the contract in this context is all but impossible, unless you know the contract language and the possible vulnerabilities that could expose your investment. Which means trusting the developer who created the contract, if they are even known¹. This has created an opportunity for Contract Audit companies to certify the contract, ensuring it does what it claims to do and nothing more. But this just moves the trust relationship one step further down the path, as in, how to know the audit company is trustworthy. And even if it is, is the audit report even understandable enough to have confidence in the contract.

The point is, with all the marketing hype around blockchain technology being Trustless, there is still a certain amount of trust in one third party or another that must be taken into account.

Carefully Manage Your Deployments

This shifted burden of trust shouldn’t be taken lightly. If the development team wants to establish itself as a trusted entity, it must take care when developing and deploying contracts. Not just by ensuring a trusted audit company has certified the contract or is able to do so, but by incorporating good development practices and processes.

Given that deployed contracts are immutable², before changes are accepted into the production version of the Smart Contract, the code should be rigorously tested and reviewed. There should also be an ability to roll back changes prior to release, quickly if needed. While there is no guarantee that a Smart Contract will be free of bugs or vulnerabilities, proper development practices and additional oversight can help reduce them considerably.

At the end of the day, when it comes to vulnerability management, a Smart Contract isn’t that different from other software systems. While DeFi is an exciting new technology, it’s important to realize that software developers in this industry have an even greater responsibility to care for their codebases. Over the years, we’ve learned that values such as pragmatism, patience, empathy, and technical excellence are some of the best ways to defend against software system vulnerabilities, no matter how old — or young — the technology is.

One of the main challenges in this space is the desire for anonymity among the parties involved. ↩
Proxy contracts notwithstanding, though they introduce another level of complexity and vulnerability. ↩

Checking in on the Digital Geneva Convention

Tue, 18 Oct 2022 00:00:00 +0000

Around 5 years ago, I wrote an article asking folks to stop treating digital weapons like biological weapons. In that article, I mentioned efforts by Microsoft to create a Digital Geneva Convention. I’d like to take a few moments to follow up on those efforts.

Multiple organizations have sprung up in the time since Microsoft made its original announcement. We’re going to take a quick glance at a few of them in this article.

Cybersecurity Tech Accord

In April 2018, a group of 34 companies, including Microsoft, announced the Cybersecurity Tech Accord with a pledge to “[protect] users and customers everywhere”. There are now over 100 companies that have signed on to the accord. This is a great first step. Getting companies, both large and small, to pledge that they are going to protect people from the abusive use of technology systems is an important milestone towards getting governments to agree not to weaponize such systems.

The Cybersecurity Tech Accord appears to be very active in its efforts to educate and bring awareness to the problems faced by our digital landscape. Their website is an excellent resource for those who want to stay on top of future developments.

Corgibytes has reached out to the Cybersecurity Tech Accord to get information on becoming a signatory.

Digital Peace Now

Digital Peace Now is a Microsoft-led effort to increase awareness and provide a platform for digital citizens to become part of the global conversation around the use of digital systems as weapons. In addition to an active blog, Digital Peace Now is hosting an online mini-lecture series called DPNU (Digital Peace Now University), and they run social media engagement campaigns.

I recently added my name to their petition, and I encourage everyone to do so.

The Paris Call

November 2018 saw the launch of the Paris Call for Trust and Security in Cyberspace. It’s a pretty simple call for limits on the use of digital systems as weapons against individuals, organizations, or governments. The call is further supported by 9 principles for governing the protection of our technological landscape and the people who interact with it.

The most recent social media campaign that’s being run by Digital Peace Now is focused on asking folks to support the Paris Call.

Corgibytes has applied to be listed as a supporter of the call, but we’re not listed on the website yet. As part of the submission process, we were asked to share a tweet-sized message describing why we were doing so.

Corgibytes is dedicated to helping organizations improve their software systems and practices. This includes improvements that address known security vulnerabilities. The need for these improvements needs to be more visible so they can receive the attention they deserve.

CyberPeace Institute

The CyberPeace Institute got started in September 2019. They are a non-governmental organization (NGO) that is dedicated to helping protect digital rights, primarily by providing services to other NGOs. As part of their advocacy work, they maintain a blog and publish analysis reports.

It’s not clear to me how people can get involved in the CyberPeace Institute outside of sending them a donation or engaging in the content that they produce.

United Nations Cybercrime Treaty

While not directly related to digital weapons, the United Nations has started negotiating a treaty to protect global citizens from cybercrime. The resolution that started this process was passed in December 2019, and negotiations are ongoing with representatives from member nations, with the first negotiating session held in February 2022.

It’s worth noting that there are some legitimate concerns about the creation of an international cybercrime treaty. The Electronic Frontier Foundation has rightly pointed out that some governments have used cybercrime as an excuse to introduce additional limitations on privacy and human rights. I’m glad that they are monitoring the development of the treaty and advocating for a narrow focus that has a reduced risk of being abused in such ways.

While not a strong rejection of the use of digital weapons by nations, getting international governments to agree that the abuse of digital systems is criminal can be seen as a step in the right direction. To say that it’s illegal for an individual, organization, or company to engage in abusive behavior can be a foundation for demanding the same of governments themselves.

I’m not familiar with the processes that govern these negotiations, so I’m not sure what to expect in terms of how long it’s going to take to get an agreement that nations are comfortable signing on to.

Others?

I’m sure there are efforts that I missed when I was researching this article. If there’s something important that you’d like to share, please do so in the comments section below.

I’m also encouraged that there has been so much movement and the creation of so many different organizations that continue to be active today. Seeing this much change in just 5 years gives me a lot of hope for what the next 5 years can bring.

The Survey Request That Improved Our Team Communication

Tue, 23 Aug 2022 00:00:00 +0000

“It’s not that I mind filling out another survey, it’s more that we haven’t heard anything about the other one we just filled out.”

The comment had no snarky tone or accusatory subtext. It was meant as a simple matter-of-fact comment on what I had just asked the team to do.

Yes, it was to fill out a survey.

Having a team member wanting to know the results of a previous survey before filling out another one was not surprising.

What was surprising was that, from my perspective, I had been updating the team about the results of the survey during our monthly strategic overviews.

For context, the monthly strategic overviews are where I provide updates on strategy, operations, and financials. Like most presentations, there is time at the end to ask questions and offer comments. We also make sure to upload the recording to Google Drive and share the link in the announcements channel so anyone who was unable to attend can view the recording. It also provides an opportunity for those who did attend to review it should they want to.

So, in case I had been tempted to believe that it was simply this one team member not paying attention during my presentations, that was squashed very quickly when others chimed in that they too would love to know the results.

What Happened?

It seems there was a breakdown somewhere between my impression of what I was saying and the reception of that information by my intended audience.

As the Collins English Dictionary would put it, there was: “a lack of communication; a failure to exchange information”.

Sure. I could have pointed to the previous recordings where I provided updates. But how would that have been helpful in the future?

Instead, I became curious and dug into the potential reasons why this breakdown happened and how to avoid it — or at least attempt to avoid it — so that I could provide information in a way that worked better for the team.

An Issue of Micro-Environment and Headspace

There are many things going on in everyone’s minds: client work, personal events, team dynamics, organizational matters, etc.

Even though I spend my days in strategy, operations, financials — and I have a pretty good memory — many times, I still need to look back at notes and agendas to confirm certain details or conversation conclusions.

And this is my world.

How then could I possibly expect others who don’t spend nearly as much time as I do in that micro-environment to remember a two bullet-point update provided once a month?

From the team’s perspective, I took them out of their daily headspace, dropped them into another one, and provided a quick note in a presentation filled with all kinds of essential information.

Despite my intent to keep everyone informed, the updates did not have the impact I thought they had.

The Solution We Landed On

For the organizational micro-environment to have a more frequent presence overall, the team suggested posting a few bullet-point highlights after each of our various weekly meetings. The rationale being that this might help provide more timely visibility into what we’re regularly discussing and advancing while keeping topics more present throughout the month.

Then, the issue of course was how can we do this without it becoming a burdensome task?

We talked about taking a few minutes at the end of each meeting to write out shareable highlights and, again, post them in the announcements channel right after the meeting.

I’ll admit I haven’t quite made this a habit and I usually end up writing mine out later. But it really doesn’t take long at all since I use the meeting agenda as a reference point.

The Experiment Continues

So far, these timely mini-updates have been well received. And they seem to both help everyone stay more informed about what’s going on in the organization and be more connected to its operational aspects.

I guess we’ll find out how well they are really working the next time I ask: “Could you please fill out this survey?”

Platform Migrations - The Built Environment as a Metaphor

Tue, 26 Jul 2022 00:00:00 +0000

Platform migrations are a common challenge among teams that we interact with. Platforms need to change frequently, to keep up with the demands and needs of the software systems they support, and the teams that use and interact with them.

Some real life examples of systems we’ve seen, and helped clients migrate, include:

Web app: Python 2.7 to 3.x upgrades
Java web app: Apache Struts 1.x to Spring Boot 2.x
Java web app: Play 1.x to Spring Boot 2.x
Rails web app: upgrade to a version that’s still receiving security updates
Windows desktop app: Win32 to WinUI

We’ve also heard of these following upgrades as highly likely candidates (we have not executed them ourselves):

Linux desktop app: X11 to Wayland
macOS desktop app: Cocoa to SwiftUI

One of the most common positions we encounter among teams that are facing this situation is that the best (or only) option is to completely level whatever they’ve got, and start over from scratch. The popular conception is that if the foundation needs to change, then the whole building needs to come down.

Construction (and destruction) as an imperfect metaphor for software

Construction has long had a place in the software industry, often used as justification for introducing more traditional engineering practices, big upfront design, and creating grand, detailed plans. The thinking has been: since software projects have been failing, if we treat them like (physical) construction projects, they’ll succeed.

Construction has nevertheless fallen out of popularity as an optimal metaphor for software development. The active assembly of a building is far more similar to compiling source code than any other activity we might do as software developers. There are many thought pieces advocating for drawing a parallel between source code and a finished blueprint (a design list of instructions). The physical construction crew takes those plans and executes them (which is what your compiler does when it takes the source code and translates it into something that can run).

The construction metaphor also fails to capture the difference between complicated and complex, especially as it relates to software systems. Complicated refers to the existence of many similarities between projects. In construction, there are a lot: even in some housing developments, almost every single house has nearly identical (or at least, very similar) floorplan and attributes, and entire neighborhoods get built that way. That consistency drives efficiency and the construction business’ ability to make more money off of repeating a process, and thus build more, faster and cheaper, leading to higher revenue (and profit). The vast majority of construction projects rely on materials that existed prior to the project starting, and very few materials must be invented from scratch to start the project. Thus, it’s possible to leverage past experience from working on previous construction projects.

In software development, almost all of these tenets break down. There are a few types of projects that don’t fit this rule (i.e. a simplified page website is relatively simple to construct, and if you’re used to using the same tool to construct it, you can avail of some of the benefits of the repetition). However, this certainly doesn’t apply to an application that must grow, evolve, and adapt in response to its users and the world in which it interacts.

When it comes to complexity, while some third party libraries exist that developers can build upon, much of the work that goes into a software project is inventing the very tools and materials that are used to produce a product. Thus, much of our past experience and performance is not usually a good predictor of our success. Because much of what we’re building is brand new, it can be difficult to predict.

Demolition is put forward as a simple idea among software teams (all you need to do is rm -rf, or Remove-Item -Recurse -Force), and then the whole project’s gone, everything’s fine, and you can just move on.

When that “simple” approach is taken, a lot of hidden issues are exposed. If a building gets torn down, a necessary precursor is that nobody is still living in it, and there’s nothing left inside. For most major building demolitions, many recyclable materials are harvested from the building first as a prep step. It’s not as simple as hitting a button, and everything is gone. The utility that the structure was providing must also be considered (and noted as an implicit opportunity cost in this approach); it cannot be providing that same utility whilst being demolished, and while something new is being put in its place. The people that were previously using that building need to move all their stuff somewhere else, and find a new building for use. Destruction (in a total demolition sense) is thus also an imperfect metaphor for software.

The built environment as a metaphor for platform migrations

While construction and destruction are highly imperfect metaphors for software, we can still turn to the built world as a fitting metaphor for software maintenance. If we envision a city as a thing that is maintained, and look at it from the top down, watching buildings coming and going, people moving throughout the city, it is noticeably more of a living creature than a static item. The built environment surrounding us is composed of tons of structures, constantly in flux. Those structures are connected via surfaces (i.e. highways, streets, and walkways, which all have their own governing rules and capabilities) that allow people to move from place to place. Materials being used in each location also need to flow through the built environment. Utilities like power, waste, water, and data are constantly traveling throughout the organism.

The built environment is a constantly changing organism, and no single plan exists that represents the precise structure of your city statically. We see the same with software systems: if you were to create an incredibly detailed blueprint that captured, in a moment, what your built environment - or your software system - is at any given time, it would immediately be out of date within moments after it was captured. Neither are static; both are constantly in movement and experiencing change.

The built environment never stops operating. Similar to the ways in which the human body continues operating throughout our lives, some parts getting damaged and healing, or changing and aging over time. There’s never a pause button that says “all work stops” - instead, anything that needs to be replaced gets isolated. For example, when a bridge is being replaced or repaired, traffic is detoured to an alternate path, not stopped completely for the duration of the work. Complete disruptions are simply not tolerated.

Structures can be thought of as a good stand-in for applications. While the initial construction of those might not be a perfect parallel, once the structure exists, the ways in which it must be maintained is very analogous to software applications. Much like any new software application, buildings have a break-in period, too. Stewart Brand’s book and BBC mini-series How Buildings Learn discusses just this: how buildings are continuously refined and shaped over time by their occupants, and start out with their fair share of kinks that need to be worked out: leaky roofs and faulty pipes, just as bugs plague software applications

Considering the surfaces that connect all of the structures around us, a decent parallel in the digital environment would be protocols that allow different applications to talk to each other. It would be thought equally unacceptable to completely stop traffic by removing a surface (i.e. road) in between two structures, just as removing a protocol would curtail the ability for two software applications to communicate with each other.

The built environment is constantly changing and adapting to meet the needs of the people around it and affected by it: new structures get built and replaced, new roads are put in, traffic lights, schools, parks, and much more.

We’ve all encountered the demolition of condemned buildings: often, making the way for some new type of structure or addition to the built environment. Similarly, it’s all too tempting (and seemingly easy) to scrap an aging and struggling software system and simply start from scratch in a quest to build something new and better. We’re fascinated by the cases in which neither of these approaches are taken, and (more in line with the needs of reality), bulldozing isn’t required to make the changes needed to the system (physical or digital).

When you want a new kitchen in the house you’ve lived in for 10 years, there’s no need to bulldoze the entire house and build it again from scratch. Instead, you can simply isolate the kitchen and renovate it. This might require you to cook somewhere else for a few weeks or order takeout, but it doesn’t require you to move all of your belongings and life out of the house for months or even years. You continue using the structure, with the specific uses adapting over time to what is available. Maybe the renovators finish the stovetop before the oven, so you can start using that first. The same concept applies to software systems. Instead of getting rid of your entire application (that might have taken years to develop and refine, and is doing its job well but has some serious issues) and rebuilding from scratch, the same type of remodeling can be undertaken in a software environment.

First, we were curious to discover how many parallels actually exist in the built environment, and how many cases this gradual transformation (move/modify without interrupting operations). A few interesting real world examples follow:

The school in Shanghai that walked to a new location: An 85-year-old primary school was entirely lifted off the ground and relocated using a new technology dubbed the “walking machine” in 2020.

The raising of Chicago: By the mid 1800s, Chicago stood at only four feet or so above the water line of lake Michigan, and as a result of being built essentially on top of a swamp, encountered huge issues with standing water and poor drainage of sewage, resulting in water-born diseases and countless other problems. Installing a sewer system was considered, though only presented a partial solution. The breakthrough plan that was eventually proposed and undertaken over the course of the next couple of decades was to lift the entire city by several meters. The clincher: operations continued in all of these buildings, with people residing in, coming and going from, and working throughout buildings that were rolling across streets and pivoting around corners. The retrospectively massive and monumental transformation that the city underwent was gradual enough that nobody even noticed they were moving, on a day to day basis.

Raising Galveston: Following a devastating hurricane in 1900, Galveston, Texas was in total ruins. Those that remained following the storm wanted to rebuild and recover from the tragedy, in a way that would hopefully mitigate the risk of it happening again. The solution devised included building a sea wall around Galveston, and in addition, raising the city up so there was a path for water to drain. The entire city was raised between two and five meters over the course of about a decade. The whole city continued to function while different pieces of it were at differing heights, many buildings still in use while being lifted. One example of how this worked was the temporary addition of elevated sidewalks that made it easier to walk between buildings.

In 1930, the 11,000 ton Bell Telephone Exchange was moved without suspending operations: in the quest to better utilize the surface area of the block where the building was located (build more offices, etc.), the original plan was simply to demolish the building. Someone pointed out that the entire region’s telephone lines ran through the building, so if it were demolished, phone service disruptions would pervade for months or more, which was unacceptable. The solution was to slide the building over, and then rotate it (specifically, over a 30 or so day period, the building was shifted 52 feet south, rotated 90 degrees, and then shifted again 100 feet west). The building remained operational the entire time - employees continued to report to work, all utilities remained hooked up, and customer phone service went uninterrupted. Workers were interviewed during and following the process, and nobody really noticed that the building was being moved throughout. Following the move, a brand new building was constructed right next to it, utilizing the space to its fullest.

Just as the CNN article of the walking school addresses decades of indifference to historic buildings, razing them to clear land for new office buildings and skyscrapers, there’s growing awareness in the software community that hard cutovers and rewrites involve more harm than anything, not to mention huge opportunity costs. Valuable architectural heritage lost as a result of demolition is akin to the inherent (and in many cases, irreplaceable) value of legacy codebases. Just as advanced technologies allow old buildings to be relocated rather than demolished, creative approaches to platform migrations can allow legacy and problematic software systems to be salvaged and repurposed, while gradually and safely transitioning to a new and more modern platform, all executed with minimal disruption to occupants and users of the system. Just as you can go on living and working in a building that is being lifted or moved, developers can go on working productively in an application that is in flux, and the application can go on serving the needs of the business.

If we draw inspiration from such events in the built environment, how might it change the ways in which we approach software development, and more specifically, platform migrations?

It is crucial to review the consequences of doing a complete demolition. Really think hard about this before proceeding. Traffic will need to be rerouted, the new platform will need to be built even while the old is still functional; how long will the switch over take, and will the new platform remain largely unused until the day it is turned on?

Is there perhaps a way to gradually start using that new application instead? For example, when building a new building next door, when the first floor is ready, could we start moving over people from the old first floor to the new first floor, so we’re able to make a gradual transition from the old building to the new, instead of waiting for the new building to be entirely finished, and then moving all sixty floors of employees, equipment, utilities, etc., over in one fell swoop?

If we can think of ways to keep the system operating as a whole - even while in motion - it can deliver a much better experience to the people paying for the work, as well as the people utilizing these applications to get their work done, because they’ll start to gradually see benefits of the new application over time, while the engineering team gets to see the benefits of not having to maintain two disparate systems. For example, when developers add a feature/update to the old system, they have to figure out how to add the same to the new system somehow and ensure it’s working correctly, even though the new system is going unused, while the old system (which will be phased out) is still (wastefully) tested and strengthened through all the use it’s getting.

When considering return on investment for your software project, consider the opportunity costs involved with any platform migration or demolition and rewrite. While all efforts are being dedicated to constructing a complete replacement (regardless of how it’s done), what other activities are being forgone, and does it make sense to dedicate budgets accordingly. Often, these platform migrations must take place sooner or later. We prefer to approach this crossroads by discovering non-destructive ways that they can be executed.

Cover photo by Pawel Nolbert.

Model Relationships in Django REST Framework

Tue, 14 Jun 2022 00:00:00 +0000

Model Relationships in Django REST Framework

I’ve spent the last couple of months working on an API written using Python, Django, and Django REST Framework (DRF). The latter is a popular, well-established framework for building APIs in Python, so I assumed it would have decent documentation surrounding a fairly common situation in any model-view-controller (MVC) framework: relational fields.

It turns out that was a faulty assumption.

Unfortunately, the documentation on working with relational fields in serializers is sparse, and many online answers and solutions are incomplete, case-specific, overengineered, or just plain wrong. It took me entirely too long to find out how to make these relationship fields to work with DRF. My solution even changed while writing this article!

In hopes of sparing other developers that pain, here’s everything I discovered about Django relational fields, and how to work with them in the Django REST Framework. In exploring this topic, I will build out a small-but-complete example, so you can experiment with these concepts yourself.

If you want to follow along, take a minute and create a Django project for yourself, with an app called pizzaria. The examples below work in Django 2.2/DRF 3.11 through at least Django 4.0/DRF 3.13, and probably beyond. If you’ve never set up a Django project before, take a look at this guide, which you can adapt to your purposes.

In addition to your favorite Python IDE, I also recommend you use Hoppscotch for crafting and testing calls to your example API, and DBeaver Community Edition for viewing your database. (Alternatively, Postman and JetBrains DataGrip work well too!)

Understanding Relationships in Django

Before we bring Django REST Framework into the mix, let’s start with an explanation of how relationships work in MVC frameworks like Django.

The first thing to know is that your Django models define how your app’s database is structured.

There are three major kinds of relationship between database entries:

One-to-many (a.k.a. a ForeignKey): for example, a Pizza is associated with exactly one Order, but an Order can have more than one Pizza.
One-to-one: for example, a Pizza has exactly one Box, and each Box belongs to one Pizza.
Many-to-many: for example, a Pizza can have more than one Topping, and a single Topping can be on more than one Pizza.

In Django, we represent these relationships as fields on models, using ForeignKey, OneToOneField, and ManyToManyField to represent them.

In the case of ForeignKey in a one-to-many relationship, you’d put that field on the model that represents the “one” side relationship: the Pizza model would have an order field, not the other way around.

When defining all three, you must specify the model that the relationship is to. See the following code; the comments will explain what different parts are for:

from django.db import models
import uuid
 
 
class Order(models.Model):
    # In this example, I'll use UUIDs for primary keys
    id = models.UUIDField(
        primary_key=True,
        default=uuid.uuid4,
        editable=False
    )
 
    customer = models.CharField(max_length=256, blank=False, null=False)
 
    address = models.CharField(max_length=512, blank=True, null=False)
 
 
class Box(models.Model):
    id = models.UUIDField(
        primary_key=True,
        default=uuid.uuid4,
        editable=False
    )
 
    color = models.CharField(
        max_length=32, default="white", blank=False, null=False
    )
 
 
class Topping(models.Model):
    name = models.CharField(
        primary_key=True,
        max_length=64
    )
 
 
class Pizza(models.Model):
    id = models.UUIDField(
        primary_key=True,
        default=uuid.uuid4,
        editable=False
    )
 
    order = models.ForeignKey(
        "pizzaria.Order",
        on_delete=models.CASCADE,
        related_name="pizzas",
        null=False
    )
 
    box = models.OneToOneField(
        "pizzaria.Box",
        on_delete=models.SET_NULL,
        related_name="contents",
        null=True
    )
 
    toppings = models.ManyToManyField(
        "pizzaria.Topping",
        related_name='+'
    )

There are a few interesting arguments on the relational fields:

on_delete: what should happen to this object when the other object is deleted? In the above, deleting an Order deletes the associated Pizza objects. However, deleting a box does not delete the Pizza. (This argument is not available on ManyToManyField.)

related_name is a sort of “virtual” field that is automatically created on the other object. (It is NOT actually added as a database column.) For example, the Box model does not actually have a contents field (and must not, to avoid colliding with the related_name here). However, within Box, I can access the associated Pizza object by accessing contents as I would any field. In the case of a one-to-many or many-to-many relationship, this will instead refer to a list of objects.

If I do not define related_name, one will be automatically created. If I set related_name to + (or end the name with +), then no “virtual” field will be created.

null defines whether the field can be null, which determines whether it is required (null=False), or optional (null=True).

There are additional parameters, which are covered in Django documentation — Model field reference: Relationship fields.

How Relationships Appear in the Database

Before we continue, it’s important to note how these relationships actually manifest in the database.

In the case of a ForeignKey or OneToOneField, a column is created. For Pizza, the table looks like this:

`Table: pizza`	id: uuid	order_id: uuid	box_id: uuid

Notice that the columns are not order and box, but order_id and box_id. The order and box fields are simulated by Django, and use these _id fields to store and retrieve the primary key of the associated object.

You’ll also notice that no column has been defined for toppings. Rather, we have a separate table…

`Table: pizza__toppings`	id: int	pizza_id: uuid	topping_id: uuid

Don’t worry, Django knows how to work with this, so you can still refer to all three via their fields.

Typically, you can let Django work out how to build this table for ManyToManyField, but if you really need control — or if you need to add additional fields to the relationship, you can specify another model on the through= parameter of the models.ManyToManyField. (See the documentation.) However, spoiler alert, a “through” model does NOT play well with DRF.

Meanwhile, in the tables for box, order, and toppings, there is no column for pizzas, contents, or the like. Django handles these related_name references outside of the database.

Serializing Relationships in DRF

In DRF, you have to define four different components to connect your model to your API endpoint:

The Model itself in models.py (or models/«whatever».py),
One or more Serializers in serializers.py (or serializers/«whatever».py),
One or more ViewSets in views.py (or views/«whatever».py),
The API endpoint specified in url.py.

The serializers are where things get interesting.

Basic Serializers

The serializer is responsible for translating data between the API request (or any other source) and the Model/database. You can use your serializer to control which fields are exposed, which are read-only, and even to simulate fields that don’t really exist.

Here is the first part of serializers.py, containing the serializers for my Order, Box, and Topping models:

from rest_framework import serializers
 
from pizzaria.models import Box, Order, Pizza, Topping
 
 
class BoxSerializer(serializers.ModelSerializer):
    class Meta:
        model = Box
        fields = ('id', 'color')
 
 
class OrderSummarySerializer(serializers.ModelSerializer):
    class Meta:
        model = Order
        fields = ('id', 'customer', 'address')
 
 
class ToppingSerializer(serializers.ModelSerializer):
    class Meta:
        model = Topping
        fields = ('name')

In each serializer, I must at a minimum specify a Meta subclass, containing the model and a tuple of fields to expose.

Handling Payload

I’ll be adding a serializer for Pizza shortly, which handles the relational fields. Before I can, however, I need to write a special function to handle the payload, the data sent with the request via the API.

Depending on how the API is used, you may receive Python objects that were deserialized from JSON or some other structured data format, or you may receive a string represention of JSON data. This gets tricky to handle on serializers!

To get around this, I created a function.py module with a function to handle the payload. It will attempt to deserialize JSON, returning the raw input if it isn’t a string representation of JSON. That way, no matter what is sent to the API, we can work with the data.

import json
 
def attempt_json_deserialize(data, expect_type=None):
    try:
        data = json.loads(data)
    except (TypeError, json.decoder.JSONDecodeError): pass
 
    if expect_type is not None and not isinstance(data, expect_type):
        raise ValueError(f"Got {type(data)} but expected {expect_type}.")
 
    return data

This code also ensures that the data type being returned is the one specified. That is, if you expect a string, the function will raise an exception if you don’t get a string. It helps cut down on silent bugs originating from the data provided via the API being the wrong type.

Serializing Relational Fields

I can now begin to write a serializer for Pizza:

from rest_framework import serializers
 
from pizzaria.functions import attempt_json_deserialize
from pizzaria.models import Box, Order, Pizza, Topping
 
# --snip--
 
class PizzaSerializer(serializers.ModelSerializer):
    order = OrderSummarySerializer(read_only=True)
    box = BoxSerializer(read_only=True)
    toppings = ToppingSerializer(read_only=True, many=True)
 
    class Meta:
        model = Pizza
        fields = ('id', 'order', 'box', 'toppings')

I have to explicitly specify which serializers to use for relational fields. We typically serialize the relational field with the serializer for the other model. This is known as a nested serializer.

For any serializer that inherits from serializers.ModelSerializer, I can pass the read_only=True parameter to indicate that I won’t be writing to the nested serializer. In most cases, DRF simply doesn’t support writing to a nested serializer.

Actually writing to a relational field is a little tricker. The best way to handle it is to write explicit create() and update() methods on the serializer. I’ll start by breaking down the create() method:

# --snip--
 
class PizzaSerializer(serializers.ModelSerializer):
 
	# --snip--
 
    def create(self, validated_data):
        request = self.context['request']
 
        order_pk = request.data.get('order')
        order_pk = attempt_json_deserialize(order_pk, expect_type=str)
        validated_data['order_id'] = order_pk
 
        box_data = request.data.get('box')
        box_data = attempt_json_deserialize(box_data, expect_type=dict)
        box = Box.objects.create(**box_data)
        validated_data['box'] = box
 
        toppings_data = request.data.get('toppings')
        toppings_data = attempt_json_deserialize(toppings_data, expect_type=list)
        validated_data['toppings'] = toppings_data
 
        instance = super().create(validated_data)
 
        return instance
	
	# --snip--

For each of the relational fields, I must intercept and interpret the payload value. That looks different for each field.

For order, I expect the UUID of an existing Order as a string. I attempt to deserialize it, in case it’s a string containing a JSON representation of a string (yes, I’ve had that happen, and it’s annoying!) I store the extracted UUID back to validated_data on the key order_id, where it will be used later when creating the Pizza object.

For box, I want to create a new Box object each time. I accept a dictionary (or string representation of a JSON object, which deserializes to a dictionary). Then, I create the new box with Box.objects.create(), unpacking the contents of the box_data dictionary in as the arguments. Then, I store the created object on validated_data on the box key.

Note: Calling Box.object.create() will bypass the serializers. If I defined create() or update() in BoxSerializer, they would never get called.

For toppings, I expect a list of topping names, which are also the primary keys for Topping objects. I wind up storing this list on validated_data on the toppings key.

As a useful aside, if I wanted to create Topping objects instead, I would use the following code instead:

toppings_data = request.data.get('toppings')
toppings_data = attempt_json_deserialize(toppings_data, expect_type=list)
toppings_objs = [Topping.objects.create(**data) for data in toppings_data]
validated_data['toppings'] = toppings_objs

Once I’ve finished refining validated_data for my use, I can create the new instance with instance = super().create(validated_data).

Interestingly, the official DRF documentation demonstrates creating the instance first, and then creating and adding items to the ManyToManyField with instance.toppings.add(). This is valid, however, I don’t like it as much because any unhandled exception at this stage will still result in the row for the Pizza being created, but with all data yet to be processed to be quietly dropped.

The update() method is almost exactly the same, except we call super().update(instance, validated_data):

--snip--
 
class PizzaSerializer(serializers.ModelSerializer):
 
	# --snip--
 
    def update(self, instance, validated_data):
        request = self.context['request']
 
        order_data = request.data.get('order')
        order_data = attempt_json_deserialize(order_data, expect_type=str)
        validated_data['order_id'] = order_data
 
        box_data = request.data.get('box')
        box_data = attempt_json_deserialize(box_data, expect_type=dict)
        box = Box.objects.create(**box_data)
        validated_data['box'] = box
 
        toppings_data = request.data.get('toppings')
        toppings_ids = attempt_json_deserialize(toppings_data, expect_type=list)
        validated_data['toppings'] = toppings_ids
 
        instance = super().update(instance, validated_data)
 
        return instance

Multiple Serializers

There’s another serializer I want: one that shows the Pizza objects on an order. I can’t just add pizzas as a field to OrderSummarySerializer, as that one is used by the PizzaSerializer, and I don’t want a circular dependency.

I’ll create OrderDetailsSerializer, which will show the pizzas on the order. To do this, I also must create PizzaSummarySerializer, which shows everything in Pizza except the order (because, again, circular dependency):

class PizzaSummarySerializer(serializers.ModelSerializer):
    box = BoxSerializer(read_only=True)
 
    class Meta:
        model = Pizza
        fields = ('id', 'box', 'toppings')
 
 
class OrderDetailSerializer(serializers.ModelSerializer):
    pizzas = PizzaSummarySerializer(read_only=True, many=True)
 
    class Meta:
        model = Order
        fields = ('id', 'customer', 'address', 'pizzas')

Creating the ViewSet

I have to write a ViewSet to expose my serializers to API endpoints, in views.py:

from rest_framework.viewsets import ModelViewSet
 
from pizzaria.models import Box, Order, Pizza, Topping
from pizzaria.serializers import (
    OrderSummarySerializer,
    OrderDetailSerializer,
    PizzaSerializer,
    ToppingSerializer
)
 
 
class ToppingViewSet(ModelViewSet):
    queryset = Topping.objects.all()
    serializer_class = ToppingSerializer
 
 
class PizzaViewSet(ModelViewSet):
    queryset = Pizza.objects.all()
    serializer_class = PizzaSerializer
 
 
class OrderViewSet(ModelViewSet):
    queryset = Order.objects.all()
 
    def get_serializer_class(self):
        if self.action in ("create", "update", "partial_update"):
            return OrderSummarySerializer
        return OrderDetailSerializer

I have a ViewSet for Topping, Pizza, and Order; there’s no sense having one for Box, since I’m only creating new boxes when creating a Pizza.

One item of note is the OrderViewSet, where different serializers are needed for different usages. When the user is creating the order, they should not have to specify the pizzas, as that relationship is handled by the Pizza model instead. However, when the user is viewing the order, they should see all the details about the pizzas.

To handle this, I define get_serializer_class(), and check for an API action of create, update, or partial_update; for any of those actions, I use OrderSummarySerializer, which does not include the pizzas field. However, for everything else, I use OrderDetailSerializer, which includes pizzas.

Creating the API Endpoints

Finally, I must expose the ViewSets via the API. In urls.py, I specify the API endpoints:

from django.urls import path, include
from rest_framework import routers
 
from pizzaria.views import BoxViewSet, OrderViewSet, PizzaViewSet, ToppingViewSet
 
router = routers.DefaultRouter()
router.register("orders", OrderViewSet, "orders")
router.register("pizzas", PizzaViewSet, "pizzas")
router.register("toppings", ToppingViewSet, "toppings")
 
urlpatterns = [
    path("pizzaria/", include(router.urls))
]

All this assumes that the pizzaria app has been configured in my Django Rest Framework project, although I’m omitting that here, since that part is pretty well documented.

Now I can interact with the API at url.to.api/pizzaria/pizzas/ and the other endpoints, and the GET and POST operations will work. I can also use url.to.api/pizzaria/pizzas/«uuid-of-a-pizza-object»/ for the PUT, PATCH, and DELETE operations.

Using the API

I sent the following payload via POST to /pizzaria/orders/:

{
    "customer": "Bob Smith",
    "address": "123 Example Road"
}

That created the base order. I received the following response:

{
    "id": "e02c46ce-742a-4656-ba9c-e80afed7304c",
    "customer": "Bob Smith",
    "address": "123 Example Road"
}

Next, I add a pizza to the order via POST to /pizzaria/pizzas/:

{
    "order": "e02c46ce-742a-4656-ba9c-e80afed7304c",
    "box": {
        "color": "white"
    },
    "toppings": [
        "sausage",
        "olives",
        "mushrooms"
    ]
}

Now if I call GET on /pizzaria/orders/, I see the pizza is on the order:

[
    {
        "id": "e02c46ce-742a-4656-ba9c-e80afed7304c",
        "customer": "Bob Smith",
        "address": "123 Example Road",
        "pizzas": [
            {
                "id": "92171c46-6526-46d0-a534-fd07fb542611",
                "box": {
                    "id": "59b84a9e-4da9-4d3e-bf5c-b157ab98f2a9",
                    "color": "red"
                },
                "toppings": [
                    "mushrooms",
                    "olives",
                    "sausage"
                ]
            }
        ]
    }
]

One mushroom, olive, and sausage pizza, coming right up, Mr. Smith!

Handling a “through” model on ManyToManyField

Our data models are working pretty well, but the API requires us to manually list all the toppings on each pizza. To save time, let’s add support for a menu. There are two considerations here:

We need to store extra information, like “medium size”.
We need to support modifications atop the menu item, like “add extra cheese”. These modifications should only affect the one order, and not the menu item itself.

In Django, we typically specify a “through” model on a ForeignKey to allow adding extra information to the field, such as in the example pizza = models.ForeignKey("pizzaria.Pizza", through=PizzaInstance) or something of that sort. However, this doesn’t work here for two reasons: first, Pizza has the key to Order, and not the other way around, and second, DRF serializers do not play well with through=!

The Models

Instead, I will create separate, standalone models for PizzaMenuItem and Pizza, where the latter represents a single instance of a pizza, associated with an order.

I’ll start this change by adding a new model for a menu item:

class PizzaMenuItem(models.Model):
    name = models.CharField(
        primary_key=True,
        max_length=128
    )
 
    box = models.OneToOneField(
        "pizzaria.Box",
        on_delete=models.SET_NULL,
        related_name="contents",
        null=True
    )
 
    toppings = models.ManyToManyField(
        "pizzaria.Topping",
        related_name='+'
    )

This is almost the same as a Pizza, except I don’t have the order field, and I’ve swapped the id for a name as a primary key, like I have for Topping.

Next, I’ll adjust my Pizza model to add a size field, a menu_item ForeignKey field, a remove_toppings field, and to rename toppings to extra_toppings…

class Pizza(models.Model):
    id = models.UUIDField(
        primary_key=True,
        default=uuid.uuid4,
        editable=False
    )
 
    menu_id = models.ForeignKey(
        "pizzaria.Pizza",
        on_delete=models.SET_NULL,
        related_name="+",
        null=True
    )
 
    order = models.ForeignKey(
        "pizzaria.Order",
        on_delete=models.CASCADE,
        related_name="pizzas",
        null=False
    )
 
    SIZE_CHOICES = Choices(
        'small',
        'medium',
        'large',
        'x-large'
    )
 
    size = models.CharField(
        max_length=32,
        choices=SIZE_CHOICES,
        blank=False,
        null=False
    )
 
    extra_toppings = models.ManyToManyField(
        "pizzaria.Topping",
        related_name="+"
    )
 
    remove_toppings = models.ManyToManyField(
        "pizzaria.Topping",
        related_name="+"
    )
 
    @property
    def toppings(self):
        toppings = []
 
        if self.menu_id is not None:
            toppings += self.menu_id.toppings
        toppings += self.extra_toppings
 
        for topping in self.remove_toppings:
            try:
                toppings.remove(topping)
            except ValueError:
                pass  # don't worry about removing absent toppings

Notice that I’ve provided a toppings property at the end. My purpose with this is to allow toppings to be added and removed atop the ones defined in the menu, without the menu item actually being modified.

I’ve also changed the toppings field to extra_toppings, and added a toppings property. When I read toppings, I want to see the toppings from the menu item and the extra toppings I added.

Models for Non-Destructive Editing

One situation where this pattern comes in handy is where you need to allow non-destructive editing. Departing briefly from our example, consider if we had a GreetingCard model, and we wanted to allow someone to modify the message on one card, without modifying the GreetingCard itself for everyone:

class GreetingCard(models.Model):
    message = models.CharField(max_length=256, blank=True, null=False)
 
class GreetingCardInstance(models.Model):
	# a ForeignKey to the GreetingCard this expands on
	card = models.ForeignKey(
		'cards.GreetingCard',
		on_delete=models.CASCADE,
		related_name='instances'
		null=False
	)
 
	# a ForeignKey to another GreetingCardInstance
	on_instance = model.ForeignKey(
		'cards.GreetingCardInstance',
		on_delete=models.CASCADE,
		related_name='modified_by',
		null=True
	)
 
	# a private field to store local data
	_message = models.CharField(max_length=256, blank=True, null=True)
 
	@property
	def message(self):
		# return the locally defined value, if there is one
		if self.message is not None:
			return self.message
 
		# otherwise, if this edits another instance...
		elif on_instance is not None:
			# delegate to that message property
			return self.on_instance.message
 
		# if all else fails, get the message from the card
		return self.card.message
 
	@message.setter
	def _(self, value):
		# store the value locally
		self._message = value
 
	@message.deleter
	def _(self):
		# delete the value locally only
		del self._message

This results in GreetingCardInstance acting like it has a message field that supports CRUD like normal, but in fact, is a bit more nuanced. The read operation checks for a value locally, on any GreetingCardInstance that this instance is intended to modify (if one exists), and then finally on the Card. However, the create, update, and delete operations occur locally only.

One interesting but important detail here is that the _message field supports both null and blank. This is usually not recommended, as it leaves the field with two distinct empty states; in this case, however, it’s actually helpful! If _message is blank, we want to treat that as an override of GreetingCard.message; however, if _message is null, we’d want to fall back on the value in GreetingCard.message instead.

The serializer coming up supports this non-destructive editing pattern, the same as it supports the primary example of the Pizza and PizzaMenuItem models.

Handling Properties in Serializers

Back to our pizzaria, the serializer for PizzaMenuItem isn’t very surprising. Here it is:

class PizzaMenuItemSerializer(serializers.ModelSerializer):
    box = BoxSerializer(read_only=True)
    toppings = ToppingSerializer(read_only=True, many=True)
 
    def create(self, validated_data):
        request = self.context['request']
 
        box_data = request.data.get('box')
        box_data = attempt_json_deserialize(box_data, expect_type=dict)
        box = Box.objects.create(**box_data)
        validated_data['box'] = box
 
        toppings_data = request.data.get('toppings')
        toppings_data = attempt_json_deserialize(toppings_data, expect_type=list)
        validated_data['toppings'] = toppings_data
 
        instance = super().create(validated_data)
 
        return instance
 
    def update(self, instance, validated_data):
        request = self.context['request']
 
        box_data = request.data.get('box')
        box_data = attempt_json_deserialize(box_data, expect_type=dict)
        box = Box.objects.create(**box_data)
        validated_data['box'] = box
 
        toppings_data = request.data.get('toppings')
        toppings_ids = attempt_json_deserialize(toppings_data, expect_type=list)
        validated_data['toppings'] = toppings_ids
 
        instance = super().update(instance, validated_data)
 
        return instance
 
    class Meta:
        model = PizzaMenuItem
        fields = ('name', 'box', 'toppings')

The serializer for Pizza requires a little more work. Since a property isn’t actually a Django field, so it is always necessary to explicitly declare the serializer that must be used with the property.

class PizzaSerializer(serializers.ModelSerializer):
    order = OrderSummarySerializer(read_only=True)
    toppings = ToppingSerializer(read_only=True, many=True)
    box = BoxSerializer(source='menu_item.box', read_only=True)
 
    class Meta:
        model = Pizza
        fields = ('id', 'order', 'box', 'menu_item', 'toppings', 'size')

The serializer specified is used to interpret the Python data returned by either the field or property; in this case, toppings works because the property is returning a list of primary keys for Topping objects.

Another interesting detail here is the box field, which I don’t define on the Pizza model! I can use the source= field to expose a field across a relational field; in this case, I get the box field from menu_item and expose that directly here as a read-only property. This requires menu_item to be required (null=False).

You’ll notice that I don’t expose extra_toppings or remove_toppings in fields. I plan to support these indirectly in create() and update(), but I don’t want to expose these fields directly; read operations should go through toppings.

Here are those two methods for the serializer:

class PizzaSerializer(serializers.ModelSerializer):
	# --snip--
 
	def create(self, validated_data):
        request = self.context['request']
 
        menu_item_pk = request.data.get('menu_item')
        menu_item_pk = attempt_json_deserialize(menu_item_pk)
        if menu_item_pk is not None:
            validated_data['menu_item_id'] = menu_item_pk
 
        order_pk = request.data.get('order')
        order_pk = attempt_json_deserialize(order_pk, expect_type=str)
        validated_data['order_id'] = order_pk
 
        extra_toppings_data = request.data.get('extra_toppings')
        extra_toppings_data = attempt_json_deserialize(extra_toppings_data, expect_type=list)
        validated_data['extra_toppings'] = extra_toppings_data
 
        remove_toppings_data = request.data.get('remove_toppings')
        remove_toppings_data = attempt_json_deserialize(remove_toppings_data, expect_type=list)
        validated_data['remove_toppings'] = remove_toppings_data
 
        instance = super().create(validated_data)
 
        return instance
 
    def update(self, instance, validated_data):
        request = self.context['request']
 
        menu_item_pk = request.data.get('menu_item')
        menu_item_pk = attempt_json_deserialize(menu_item_pk)
        if menu_item_pk is not None:
            validated_data['menu_item_id'] = menu_item_pk
 
        order_pk = request.data.get('order')
        order_pk = attempt_json_deserialize(order_pk, expect_type=str)
        validated_data['order_id'] = order_pk
 
        extra_toppings_data = request.data.get('extra_toppings')
        extra_toppings_data = attempt_json_deserialize(extra_toppings_data, expect_type=list)
        validated_data['extra_toppings'] = extra_toppings_data
 
        remove_toppings_data = request.data.get('remove_toppings')
        remove_toppings_data = attempt_json_deserialize(remove_toppings_data, expect_type=list)
        validated_data['remove_toppings'] = remove_toppings_data
 
        instance = super().update(instance, validated_data)
 
        return instance

Serializers for Non-Destructive Editing

I’d like to briefly revisit the greeting card non-destructive editing example.

The serializer for GreetingCard is as simple as it comes. Since message is just a CharField, a Django primitive field, I don’t need to do anything special to serialize that field:

class GreetingCardSerializer(serizaliers.ModelSerializer):
	class Meta:
		model = GreetingCard
		fields = ('message',)

The serializer for GreetingCardInstance requires a bit more work. Because message is a property, I still have to explicitly specify the serializer to use, despite the fact the property returns a value from a primitive Django field type:

class GreetingCardInstanceSerializer(serializers.ModelSerializer):
	message = serializers.CharField(max_length=256)
 
	class Meta:
		model = GreetingCardInstance
		fields = ('message',)

The CharField serializer accepts the same arguments as the field in regard to what the data looks like. That’s always the case for primitive serializers: decimal places, length, choices, and the like, all have to be defined on the serializer. I like to think of these as defining the “shape” of the data, which both a field and a serializer would need to know. However, it is not necessary to specify whether null or blank are allowed, nor do you have to repeat the verbose name or other such field metadata.

Admittedly, it gets a little annoying to have to duplicate the arguments defining the “shape” of your data, but it’s worth the extra effort to get this non-destructive editing.

WARNING: If you try to create an object, like GreetingCardInstance, via GreetingCardInstance.objects.create(), it will bypass the serializer! As such, you would not be able to include a value for message in that call; you’d have to specify a value for _message instead.

Views, URLs, and API usage

Returning once again to the pizzaria example, my update to views.py is not surprising in the least:

# --snip--
 
class PizzaViewSet(ModelViewSet):
    queryset = Pizza.objects.all()
    serializer_class = PizzaSerializer
 
 
class PizzaMenuItemViewSet(ModelViewSet):
    queryset = PizzaMenuItem.objects.all()
    serializer_class = PizzaMenuItemSerializer
 
# --snip--

Warning: Some solutions online suggest using your ViewSet objects to manipulate the create and update payloads before they reach the serializer, but I strongly advise against this. The further the logic is from the model, the more edge cases the logic is going to be unable to elegantly handle.

Finally, I expose the new menu endpoint in urls.py:

# --snip--
 
from pizzaria.views import OrderViewSet, PizzaViewSet, ToppingViewSet, PizzaMenuItemViewSet
 
router = routers.DefaultRouter()
router.register("orders", OrderViewSet, "orders")
router.register("pizzas", PizzaViewSet, "pizzas")
router.register("toppings", ToppingViewSet, "toppings")
router.register("menu", PizzaMenuItemViewSet, "menu")
 
# --snip--

Now I can make some API calls. On the api/pizzaria/menu/ endpoint, I call POST with the following payload:

{
    "name": "ansi standard",
    "toppings": [
        "pepperoni",
        "mushrooms"
    ],
    "box": {
        "color": "black"
    }
}

Now on api/pizzaria/pizzas/, I call POST with the following payload:

{
    "order": "017eae04-5123-41ac-b944-8f8208d75298",
    "menu_item": "ansi standard",
    "size": "large",
    "extra_toppings": [
        "sausage",
        "olives"
    ],
    "remove_toppings": [
        "mushrooms"
    ]
}

You’ll notice that I specify extra_toppings and remove_toppings as lists of strings (primary keys for Topping objects).

Calling GET on the same endpoint shows me the following:

{
    {
        "id": "166de02c-3753-46db-8cab-451ad0be5a4a",
        "order": {
            "id": "017eae04-5123-41ac-b944-8f8208d75298",
            "customer": "Bob Smith",
            "address": "123 Example Road"
        },
        "box": {
            "id": "5f5aafd6-352d-48e9-926a-25d7bb0be9f9",
            "color": "black"
        },
        "menu_item": "ansi standard",
        "toppings": [
            {
                "name": "olives"
            },
            {
                "name": "sausage"
            },
            {
                "name": "pepperoni"
            }
        ],
        "size": "large"
    }
]

Notice that toppings is displayed, but extra_toppings and remove_toppings are absent. Remember, the “ansi standard” pizza is pepperoni and mushroom, but on this pizza, I asked to remove mushroom (leaving only pepperoni) and to add olives and sausage. Thus, I see pepperoni, which is coming from the PizzaMenuItem object, and olives and sausage from the Pizza object. Mushrooms are omitted because of how the toppings property uses remove_toppings on the Pizza object.

I also see the box field here, which is coming from the PizzaMenuItem object; it never existed on the Pizza object.

Summary

Despite the many surprises DRF presents to the developer when working with relational fields for the first time, once you understand the patterns, it’s not terribly difficult.

Define relationships in your models with ForeignKey, OneToOneField, and ManyToManyField fields.
Use properties on your models to simulate fields that store and interpret data from related models.
Use your serializers to intercept and interpret data meant for these relational fields, to expose properties as fields, and to expose fields on related models.
Use viewsets to expose these serializers. Avoid using a viewset to manipulating the payload being passed to the serializer.

Happy coding!

Empathy System Architecture

Mon, 18 Apr 2022 00:00:00 +0000

If you’re a software developer, there’s a skill that is becoming increasingly important to your work. Empathy. Understanding other people’s feelings and experiences helps us meet the needs of the people who will use what we create as well as writing code in a way so that it’s easier to maintain. That’s why “Act With Empathy” is our first core value at Corgibytes.

However, there’s a HUGE gap in understanding. This isn’t because software developers lack empathy — far from it and we should fiercely combat any idea that reinforces that stereotype. Instead, I think the problem is that most of the people who understand empathy deeply don’t know how to code, so they’re not able to make the connections that would make empathy concrete and applicable to their daily work.

Over the years, as I’ve spoken about empathy in software at conferences, I’ve been repeatedly asked some form of this question:

“I’m a back-end software engineer. I don’t interact with the customer. I don’t work on the user experience. So, where, exactly, does empathy show up in the work I do? How does empathy relate to things like databases and object-oriented languages?”

Lately, I’ve been diving deep into research to answer this question. The more I learn, the more I see that software and empathy have much more in common than we might think. Since I spent a decade in communications understanding empathy and a decade in software leading Corgibytes, maybe I can help be a translator between these two worlds.

Empathy as a Set of Interconnected Skills

First, we need to get clear on what empathy even is. Recent neuroscience is indicating that it’s not a feeling or character trait. Instead, it’s a set of different skills that can be developed and learned. Here’s the definition I find to be the most complete.

Empathy is a system of interconnected skills that are used to accurately and compassionately understand and help people.

For the past year, as part of the research for my upcoming book, Empathy-Driven Software Development, I’ve been hunting for exactly which skills are part of this system. As I began to piece together the puzzle from reading hundreds of research papers and books, I started to see that empathy has an underlying architecture similar to software, which I’m calling Empathy System Architecture.

With Empathy System Architecture, we can map out the fundamental skills required for empathy in the context of building software as a way to facilitate learning. Adults learn by associating new concepts to past experiences. By linking empathy to computing concepts you might be more familiar with, it might make integrating these new concepts easier.

So, let’s dive in, shall we?

Senses are like Empathy Inputs

The human body is equipped with specialized neurons that enable us to interact with the world around us through senses. This sensory energy is then converted into electrical signals known as neural impulses that are sent to the brain to be interpreted. Sensing using our neurons has some similarity to how a computer might receive information from a keyboard, touchscreen, or mouse. The physical energy from the click is transformed into electrical signals that can be further processed.

As we develop empathy, we need to pay attention to sensory information. These signals can come from our external environment, such as what we see, touch, hear, smell, and taste. However, there’s also much more sensory information that our bodies process, such as what is going on internally, which scientists call interoception. Our brains are constantly monitoring levels of pain and fatigue, temperature, hunger, thirst, metabolism, breathing, heart rate, and even circadian rhythm.

Similar to how we learn to spot patterns while we code, learning to pay attention to sensory signals is an important skill for empathy.

Perception is like Empathy Processing

Similar to how the electronic circuitry in a computer executes a computer program, our brains use perception to select, organize, interpret, and cognitively experience sensory information. Perception is what enables us to assign meaning to the neural impulses we generate.

While comparing our brains to the Central Processing Unit of a computer can work as an abstracted and generalized metaphor, this metaphor starts to break down once we dive a bit deeper. So, when it comes to perception, it can be useful to extend the metaphor to include production code and not just the CPU.

For example, recent neuroscience research is showing that we instantiate emotions in a remarkably similar way to how we might instantiate a class at runtime (also known as duck typing.) When our bodies detect a sensory pattern, we’ll give it a name. For example, when I notice that my heart rate increases, my mouth is going dry and my breathing becomes more shallow, I’ll take those attributes and assign a name to them, such as nervous or excited, depending on the context.

Once we’ve named a concept, our brains will then call behaviors that have worked in the past, similar to how object-oriented languages use methods to determine the behavior of a class. When we think of perception as production code, we realize that we have the power to change our behavior in the moment. For example, I tend to talk faster and louder when I’m nervous. If this isn’t the behavior I want, I can override it similar to how we might throw an exception or use monkey patching.

The ability to adapt our behavior to our surroundings is an important part of empathy. It allows us to respond compassionately to what other people need.

Communication is like Empathy Output

Sharing what we perceive is an important part of developing empathy. When we communicate, we are transmitting information. What’s amazing is that the exact same communication model is the foundation for sending information between computers and between humans.

Shannon-Weaver model as presented in A Mathematical Model of Communication.

The Shannon-Weaver model of a general communication system was developed in the 1940’s by mathematician Claude Shannon when he was working at Bell Labs. A message begins at an information source, gets constructed with a transmitter, is sent via a channel, is interpreted by a receiver, and concludes its journey at a destination. At this point, the message gets processed at the destination and becomes part of a new message that is sent back to the original source, creating an ongoing loop that continually processes information.

Seeing this model for the first time in the context of writing software blew me away because it seemed so familiar. As it turns out, this is the exact same model that was used in many of my marketing and communication courses in college. Not something that was inspired by the original. Not a derivative model. It was this schematic. The exact same one that made the internet and all forms of digital communication possible. That’s because people like Warren Weaver and John Robertson Pierce took Shannon’s math and added context that was valuable to people who weren’t telecommunications specialists.

Now, this model has been criticized by some people in the social sciences, but I think in many ways that’s because it’s been generalized. If we dive deeper into the mathematical principles, we can explore how to construct a message so that it is clear and will likely be understood by a given audience. Using this model can also help us anticipate problems with how our message might be received.

For example, entropy, or the amount of statistical uncertainty, is a key principle of Shannon’s mathematical model. You may have come across this principle if you’ve done work in cybersecurity because high entropy is essential for generating cryptographic functions, such as ssh keys, that are difficult to break.

Entropy is related to another aspect of Shannon’s model — compression. Messages in a low-entropy (high certainty) system can more easily be compressed than ones in a high-entropy (low certainty) system. In daily communication, this means that we can predict the amount of entropy between ourselves and another person. The lower the entropy (higher certainty), the more linguistic shortcuts we can use.

Here’s how this might work in your day-to-day work. Let’s say that you’re collaborating with a teammate who has similar experience to you and you’ve been working on the same project together for several years. This experience gives you a high degree of confidence that your pairing partner would accurately interpret this alphabet-soup of a sentence. (Note: if you don’t know what this sentence means, don’t worry. You don’t need to.)

“We’re using ASP.NET MVC but don’t put the logic in the UI. That makes it easier for the marketing team to change the HTML and URLs for SEO. Plus, it’s easier for us to do TDD.”

The reason you can use these acronyms with confidence is because there is high certainty (low entropy), you can take advantage of compressing bigger concepts into letters without compromising the probability that your message will be understood.

Keeping this kind of probability in mind can help us construct better messages with people who we don’t know very well, too. For your colleague in the marketing department, you can probably guess that they wouldn’t know what most of the acronyms mean. So you might convey the same message in a different way.

“It’s tempting to want to put the logic right alongside the code that controls what you’ll see on the page. But there are some good reasons we should separate them. That will make it easier to make changes later, so we should be able to add new features faster. It’s also easier to test, so the risk of introducing a bug or crashing the app is lower. Plus, you’ll have lots more control of the links and our app would get better rankings on search engines.”

While this second paragraph is less efficient in terms of word count, it’s likely much clearer to the person you’re talking to who doesn’t code.

Artifacts are like Empathy Storage

As we sift through a codebase, we can find evidence of other people’s thinking. This is often key to being able to quickly build a mental model of what a system does. We can also reverse this thinking. While we’re coding, we can imagine people in the future and leave something for them to find.

In this way, codebases can become a type of time capsule. Just how an archaeologist or anthropologist might use artifacts and context to understand a past civilization, there are many opportunities in our daily development practices to leave artifacts of our intentions for another person to reconstruct.

When we sit down and think about the number of artifacts we create while we write software, it’s pretty staggering. With workshops I’ve run, there are always piles and piles of sticky notes, often to the surprise of the participants. Here are some categories and examples of artifacts to consider while you code. With each of these, a bit of empathy during their development can go a long way to reducing the frustration of someone who comes after you (which may even be your future self.)

Language elements - functions, classes, methods, variables, subroutines, etc.
File structure - READMEs, /docs/*md, directory and file names, directory hierarchy
Version control - commits, pull/merge requests, change logs
Automated tests - Unit tests, integration tests, acceptance tests, approval tests, scenarios and specifications (ex: Gherkin)
Third-Party dependencies - APIs, Linters, databases, libraries, packages,
DevOps - deployment tools, analysis tools, CI/CD tools, environment setup, orchestration, observability
Generated Bi-Products - Logs, WAR files, output, reports, graphs
Contextual documentation - wikis, code quality reports, product guides, knowledge bases, process guides, onboarding guides, exploratory test guidelines,
Customer/User communication - product documentation, user guides, training, FAQs, marketing websites, social media, release notes, alt tags
Project communication - backlog, issue tracking, story/epic tracking, burndown charts, budgets, timesheets, architecture decision records, departmental reports,
Personal communication - social conversation, interpersonal conflict, negotiation, emotional support

It’s easy to slip into a complacency when creating some of these artifacts. You might even find yourself thinking “who’s even going to read this?” When that’s the case, pause and ask yourself who will read what you create.

For example, I used to think no one really used alt tags, so early in my career I’d kinda blow them off. Then my friend Taylor lost his vision in a car accident and he told me just how important alt tags were to him. That changed everything. Now, because I care about Taylor, I want to do what I can to make his experience with what I create as pleasant as possible. Identifying the human and empathizing with their experience can help us find the motivation to create more maintainable and accessible software.

Boundaries are like Empathy Security

Boundaries are absolutely critical to creating systems that are healthy and resilient. You don’t have to look far in the software world to see this. Many programming patterns are rooted in setting and managing boundaries between different system elements (encapsulation, strong cohesion, loose coupling, microservices, bounded contexts, automated testing, and SOLID principles are just the tip of the iceberg.) In general, healthy boundaries = healthy codebases.

The same can be said for people. Healthy boundaries = healthy relationships. However, many of us (myself included) have been conditioned to believe that operating from a place of empathy means always acquiescing to other people’s needs.

That’s not empathy. That’s being a doormat.

Compelling research on this comes from Adam Grant’s book, Give and Take. People who help others with no strings attached tend to be highly successful, but only if they maintain strong personal boundaries. His research shows that people who don’t “give their time and energy without regard for their own needs, and they pay a price for it. Selfless giving, in the absence of self-preservation instincts, easily becomes overwhelming.” Operating without personal boundaries actually hurts our capacity to empathize with others.

What do healthy boundaries look like? In her book Set Boundaries, Find Peace, Nedra Glover Tawwab provides three categories:

Porous - when we don’t clearly communicate what’s okay and what’s not okay, it can create an unhealthy closeness (enmeshment) that can drain our energy and lead to unhealthy relationship dynamics.
Rigid - if our boundaries are too inflexible, we can inadvertently push people away which prevents us from developing the trust required for a deep connection.
Healthy - boundaries that enable resilience “require an awareness of your emotional mental, and physical capacities, combined with clear communication.” You are able to give without giving too much, which often requires saying no with confidence.

To help us understand how to set and maintain healthy boundaries, we can turn to the world of cybersecurity. In 2015, Symantec published a whitepaper titled “The Cyber Resilience Blueprint: A New Perspective on Security” where they advocated that “organizations must change their security posture from a defensive stance focused on malware to a more realistic and resilient approach—a cyber resilient approach.” In other words, porous boundaries were bad, but this was a recognition that rigid boundaries could be equally bad. Symantec’s “Five Pillars” which describe healthy security policies also happen to mirror healthy boundary policies.

Prepare/Identify - First, we need to identify what we need to protect. This means having compassion for ourselves and being honest about the boundaries we need to uphold, even boundaries that we need to hold ourselves accountable for.
Protect - There are six types of boundaries that we need to monitor, according to Twwab: physical, sexual, intellectual, emotional, material, and time. The goal is to communicate boundaries in a way that is clear, direct, appropriate, and consistent.
Detect - People will bump up against your boundaries constantly. The ability to notice violations is important. Seldom do you establish a boundary just once. It’s an ongoing process.
Respond - When a violation occurs, it’s important to resolve the issue promptly to prevent boundaries from becoming porous. Behavioral regulation and consistency is key so that our actions serve our goals.
Recover - Self-care and reflection are an important part of the boundary setting process. It can be helpful to proactively develop healthy remediation and recovery strategies, such as therapy, exercise, or spending time in nature, that might help you heal when a boundary violation occurs.

Relationships are like Empathy Networks

There are many ways that we can connect computers to each other. The arrangement of nodes and their connections is just as relevant with people as it is with computers. In the field of telecommunications, we use the term network topology. When we’re analyzing social systems, it’s referred to as social network analysis. Whether you’re looking at people or computers, the goal is to map out relationship structure to understand how information flows through a system.

Using nodes and links, you can map out how both computers and people form relationships with each other. Image Source: Adobe Stock/kytalpa

For example, as an individual, it’s useful to observe how you relate to the people around you. Some of your relationships will be stronger than others because there has been more opportunity to build trust. For example, your relationship with a trusted long-term friend is stronger than that of the cashier at your coffee shop.

Looking at your personal network topology is helpful for your personal well-being as it can help you figure out where to spend your time and energy investing in relationships. There are some relationships you might want to strengthen (say, a new co-worker who is working on the same project as you) and which ones you may want to weaken (perhaps a person who is consistently disrespectful and doesn’t honor your boundaries.) It can also help you discover a community and build social capital that can help you reach your personal or professional goals.

From an organization perspective, interaction topology is important for productive teams, a concept that Matthew Skelton and Manuel Pais explore in their book Team Topologies. We can also see this in open-source projects on GitHub, where research has found that networks that placed a high emphasis on rich interactions and clustering were more productive and resilient than those with a hierarchical structure with only a few key maintainers. Finally, paying attention to your team’s communication architecture matters because, according to Conway’s Law, it will be mirrored in your system architecture, too.

Attunement is like Empathy Synchronization

In software, we have many different protocols that help us ensure that information is synchronized between different devices. For example, TCP/IP is largely what runs the internet. Messages are broken up into packets, sent across a network, and then reassembled on the other end. The emphasis of this protocol is accuracy. Did the packets, which may have each taken many different routes to arrive at its destination, eventually arrive in the correct order?

For human relationships, attunement is the mechanism that allows us to build trust through accurate understanding. At its core, attunement is about listening without judgement or defensiveness. When we attune, we are intentionally making an effort to synchronize our understanding with another person’s emotion and experience. We are doing our best to ensure the packets of information that another person is sending are being received correctly on our end.

Attunement is a choice. Do we turn towards the opportunity to understand or away from it? This choice happens in small day-to-day moments, bigger more significant moments, and also in times of conflict. In his book, The Science of Trust, relationship researcher John Gottman dives deep into the theory of attunement, which he developed with his graduate student Dan Yoshimoto. “The skills of attunement,” Gottman explains, “can make all the difference between a relationship’s strength versus its demise.”

Perspectives are like Empathy Schemas

In both software development and psychology, models and schemas are terms that we use to describe how we organize information and explain conceptual relationships. Our brains hold an extraordinary amount of data that we use to make sense of the world. As we navigate our experience, our internal database is constantly updating itself and reorganizing information to help us conceptualize reality. It’s as if we have a massive database inside our minds. It’s our perspective — a dataset of experiences that is unique from everyone else.

In psychology, schema theory was first developed by Jean Piaget in 1923 and is a foundation for popular therapeutic techniques of today, such as Cognitive Behavioral Therapy. Schemas are heuristics that we’ve built in our minds so that we can rapidly and intuitively assign meanings to things.

For example, how do you identify if something is a table or a desk? In many ways, their attributes are the same. They both are commonly rectangular, made of wood, are similar heights and have four legs. The differences might be subtle, but we learn to distinguish them through lived experience. In other words, we’ve built schemas around them.

Schemas aren’t just for tangible objects either. We form concepts to categorize abstract things, too. Colors, ideas, and emotions — all of these are schemas that are built on our internal mental models. So are concepts that we might want to change, such as stereotypes.

If we want to operate with empathy effectively, it’s important for us to find ways to enable other people’s perspectives to become compatible with our own. While this can be a challenge, there are tools that we use to build software that can help us. In particular, we can look at relational databases.

A relational database consists of different tables, each which are made up of rows and columns like a spreadsheet. The database is built by creating relationships between these tables, which makes the data relatively easy to retrieve and modify.

As developers, we can use tools such as Structured Query Language (SQL), Class Diagrams, and Entity-Relationship Models to help us wrap our brains around what’s going on. We can adapt these tools that we already use to help us map out our perspectives. Then, we can work on creating ways to easily interface with other people, similar to how we might build or consume an API.

While this might seem far fetched, consider that researchers and developers in the field of Natural Language Processing are already using these types of techniques to personalize the responses of computer systems in programs such as chatbots. You can also check out John Sawers, who talks about Emotional APIs.

Decisions are like Empathy Algorithms

As humans, we build our own algorithms and formulas to help us evaluate information and choose a course of action. In the book Algorithms to Live By: The Computer Science of Human Decisions, computer scientists Brian Christian and Tom Griffiths explore what they call “human algorithm design” which is about applying computer science principles in ways that help humans find better solutions for every day decisions.

This matters, they argue, because “Thinking algorithmically about the world, learning about the fundamental structures of the problems we face and about the properties of their solutions, can help us see how good we actually are and better understand the errors that we make.”

The decisions we as technologists make ultimately end up in the technology we build. This is especially prescient in the field of machine learning, where biased datasets are increasingly being used to determine who gets an interview for a job, who qualifies for a mortgage, who gets granted parole, which results come back for search terms, and what information gets shown on social media and online advertising. These seemingly small decisions have a big impact on an individual’s life, impacting their earning potential, civil rights, safety, health, and well-being.

Dr. Joy Buolamwini, founder of the Algorithmic Justice League defines this problem as “coded gaze” a term which she developed to describe “the algorithmic bias that can lead to exclusionary experiences or discriminatory practices.” According to Buolamwini, coded gaze is “a reflection of the priorities, preferences, and also sometimes prejudices of those who have the power to shape technology.”

Being aware of how we make decisions and consistently trying to make more compassionate choices is an important aspect of empathy.

Ethics are like Empathy Integrity Constraints

“The scientist, engineer or manager may well wash his hands but this will not free him from moral duties or social responsibilities — not only qua a human being and a citizen, but also as a professional. And this because, let us recall, they more than any other occupational group are responsible for the shape the world is in. You cannot manipulate the world as if it were a chunk of clay and at the same time disclaim all responsibility for what you do or refuse to do, particularly since your skills are needed to repair whatever damages you may have done or at least to forestall future such damages. In short the engineer and the manager, precisely because of the tremendous power they wield or contribute to building up, have a greater not a diminished moral and social responsibility. This being so, they had better face it.” - Mario Bunge, “Towards a Technoethics”, 1977

As technologists, it’s important to recognize that ethics is a critical part of the work we do. This means consistently considering the impact that our work will have and modifying our actions accordingly. The optimism bias is seductive, but we must think beyond whether or not something could be built and think also about whether or not it should be built. While it’s tempting to dismiss ethics as something that can be delegated to another department, as software developers, we cannot abdicate our responsibility, as the philosopher Mario Bunge points out in the quote above.

Again, we can look at how we work with data in software to help us explore ethics and empathy. In a relational database, a key aspect is what’s called an integrity constraint. These are rule sets which are put in place to make sure the data stays accurate, consistent, and reliable. In a similar way, ethics are rule sets we embed into our decision making processes to help us weigh whether our actions are moral, fair and just.

For example, the Ethics and Compliance Initiative recommends thinking about problems in through what they call the four “PLUS filters.”

P = Policies - Is it consistent with my organization’s policies, procedures and guidelines?
L = Legal - Is it acceptable under the applicable laws and regulations?
U = Universal - Does it conform to the universal principles/values my organization has adopted?
S = Self - Does it satisfy my personal definition of right, good and fair?

Then, while you are making decisions throughout your daily work, you go through the following seven steps:

Define the problem (consult PLUS filters)
Seek out relevant assistance, guidance, and support
Identify alternatives
Evaluate the alternatives (consult PLUS filters)
Make the decision
Implement the decision
Evaluate the decision (consult PLUS filters)

This is just one of many different ethical frameworks out there. Incorporating ethical considerations into our daily decision making is an essential aspect of coding with empathy.

Compassion is like an Empathy Power Source

Imagine you own a car. If you want to use it, you’ll need the right fuel. Otherwise, you won’t get very far. Such is the relationship between empathy and compassion. Empathy is like the car and compassion is like the fuel. Or, if we’re sticking with our computing metaphor, empathy is like the laptop and compassion is like the battery.

While there are many overlapping qualities of empathy and compassion, the distinctive quality of compassion is the active desire to relieve suffering. Research from the Center for Compassion and Altruism Research and Education (CCARE) at Stanford University describes compassion as having four key components:

Awareness - cognitively recognizing suffering
Affect - being emotionally moved by suffering
Intention - wanting suffering to cease
Motivation - a readiness to help relieve suffering

Compassion is cultivated and nurtured with sustained practice, which you can learn through programs such as Compassion Cultivation Training, which was developed through the researchers at Stanford’s CCARE. Specifically, this includes skills such as learning to slow down and pay attention (mindfulness), generating feelings of caring and warmth (loving-kindness meditation), and increasing compassion for yourself (self-compassion).

In this way, compassion becomes enduring and active. Compassion isn’t something that happens to you. Rather, by adopting a compassionate mindset and purposefully nurturing it, your capacity for compassion can increase over time.

Empathy that isn’t fueled by compassion can become taxing and toxic. Without compassion, attempts to understand and help people can backfire and often aren’t sustainable. For example, if we have compassion for everyone else, but forget ourselves, our poor boundary setting can lead to burnout. If someone is able to cognitively understand another person, but doesn’t care about how they feel, they’ll be more likely to manipulate them or inflict harm. On a group level, having compassion for some people but not for others can lead to tribalism where entire groups of people are disparaged or dehumanized, sometimes based on a single characteristic (race, ethnicity, nationality, gender, etc.)

When our compassion tanks are full, we have the energy to do the hard work of perspective-taking, attunement, boundary setting, communicating, and the other skills in Empathy System Architecture. Without compassion, we can expect an empathy breakdown.

Prosocial Action is like Empathy In Production

A software production environment is where the latest version of the code is made available for use. Infrastructure, servers, source code, databases, and more all work together to deliver software that, hopefully, works as it was intended. Software in production is software in action.

This is true for empathy, too. Empathy requires interaction. It doesn’t just live in our minds. This is similar to how we might write code on our local environment, but to make it useful, we need to deploy it to production. As the old proverb says, “The road to hell is paved with good intentions, while the road to heaven is paved with good works.” It is our action that matters, not our intent.

Eek! If this sounds scary, you’re not alone. Operating with empathy means that we’ll make mistakes. We’ll mess up. We’ll blunder and get things wrong. Empathy is a vulnerable process where we are consistently facing our own inadequacies. Learning to stick with it and work through the big challenges is part of the package.

This happens when we write code, too. Bugs crop up that we didn’t expect. A change to a dependency introduces a new security vulnerability into our system. We bang our heads against the keyboard trying to solve a tough problem. But we keep going. We learn. We patch things up as best we can and try to make things better.

Empathy isn’t about being perfect, but it is about orienting our actions so that we are more likely to help than harm. When empathy goes well, we can see higher trust, more joy, more creativity and innovation, and higher productivity — all because we’re able to work better together.

Empathy Training for Software Teams

Believe it or not, what you’ve read only scratches the surface of many of these ideas. Just like you can go incredibly deep on a single aspect of software system architecture, there’s a rich treasure trove of information about any aspect of empathy system architecture, too.

If you found this article interesting, be sure to sign up for updates about my book at empathyintech.com. We’re also hosting events and have an active Discord server to help make empathy training in the tech industry as accessible and inclusive as we can.

And finally, while empathy skills are absolutely a key part of what makes Corgibytes so good at software modernization and maintenance, we’ve determined that teaching these concepts to other teams is outside of our core competency. This means that you’ll be hearing a bit less about empathy from me on the Corgibytes blog, but…

…(drum roll)…that’s why we’ve launched Heartware — our sister company that will focus entirely on helping developers build these critical skills in the context of their daily work. While I’m still actively involved in Corgibytes, my main focus for the next phase of my career will be building this new business that helps turn “soft” skills into software skills.

Over the past decade, I’ve seen first hand how actively making empathy part of your daily development practice can have a profound positive impact. Empathy System Architecture helps take empathy out of the clouds and into the code. When we look at empathy as an integrated system rather than as a collection of isolated components, we can start to see how elements interact and decide where we want to focus our attention.

The Whys and Hows of Applicant-Centered Recruiting at Corgibytes

Thu, 27 Jan 2022 00:00:00 +0000

Have you ever filled out a job application only to never hear back from the company? Probably more than once. Probably more than ten times, right? It seems like the cultural norm is to ignore job applicants - unless you’re calling them in for an interview.

Let’s be clear. This is awful. It can feel so dehumanizing to be ghosted and ignored. Applying for a new job is an inherently vulnerable process. Putting yourself out there isn’t easy. So when companies don’t even acknowledge the emotional labor you put in, that stress can compound. We know because most of us have been there ourselves.

Recently, I’ve taken up the mantle of managing the recruiting process that Andrea and Scott originally developed. You can read all about the in’s and out’s of our recruiting process by visiting the hiring process page of our website. We go into great detail about the number of interviews, what the process looks like, etc. That’s the function of our recruiting process. I want to talk about the philosophy of it in this blog post. Our process is different, but to me, that’s good. Recruiting, especially in the tech industry, is broken. Too many companies use practices that harm workers and don’t actually help them find the best candidates.

Problematic but Popular Recruiting Practices

Let’s take a look at some of the biggest flaws we see in recruiting practices today to help give you context into why we approach recruiting the way we do.

Personality Tests

Assessing a candidate’s personality using a survey tool is an incredibly common step in the interview process. However, we think the problems outweigh the benefits. According to disability justice advocate Lydia XZ Brown, “Personality tests are by and large constructed to be ableist, to be racist, to be sexist, and to be classist” because they are often “…based on norms devised from college-educated straight white men with no known disabilities. Personality tests are useful for individual people sometimes on journeys of self-discovery. But when they’re used to make decisions by other people affecting someone’s life, they become dangerous tools.”

Andrea and Scott both have invisible disabilities and experienced this challenge first-hand, so they felt strongly that any survey tool we use that measures someone’s characteristics should: 1) only be used after someone is hired, 2) be a tool that has been validated by research, and 3) be one of many informing factors and not a deciding factor. While an estimated 60%-70% of American workers are required to take a personality survey that is uncorrelated with performance, we choose to keep surveys like these out of our recruiting process.

Automated Resume Scanning

According to recent research by the Harvard Business School, there are an estimated 27 million “hidden workers” in the United States. These are people who are highly qualified and want to work, but “experience distress and discouragement when their regular efforts to seek employment consistently fail due to hiring processes that focus on what they don’t have (such as credentials) rather than the value they can bring (such as capabilities).”

One major contributor is using automated tools that are designed for efficiency more than effectiveness. Practices such as scanning resumes for keywords and other specific parameters (such as having a college degree or not having a gap in full-time employment) are often used as the initial step to whittle down who gets a call back for an interview. The problem here is that this practice evaluates candidates by proxy instead of potential performance. When this research came out, we were relieved to see that many of the recommendations on how to prevent this problem had already been part of our hiring DNA from the very beginning. Just because more than 90% of companies approach recruiting this way, doesn’t mean we have to.

“Cultural Fit” Biases

If you are gregarious, excitable, and assertive, research shows that you have an edge when it comes to landing a job. This isn’t necessarily because of cognitive ability or because you are a better fit for the role. It’s because you benefit from society’s (particularly American society’s) strong preference towards people who are extroverted.

However, research shows that being deliberate and quiet has its advantages in the workplace and that in many contexts, introverts tend to be better leaders, even in sales positions, where it is assumed that extroversion is a must-have trait.

You don’t need to be an extrovert to be a good communicator, have empathy, or work with technical excellence. But many organizations inadvertently design their recruiting process to inadvertently put introverts, or other people who may have different styles of expressing themselves and connecting with people, at a severe disadvantage.

In our opinion, there is far too much weight placed on things like body language, eye contact, verbal speech patterns, accents, wardrobe, or perceived levels of excitement. These often get lumped into a vague category of “cultural fit,” which can lead to discrimination and are often the result of confirmation bias more than competency. Instead of trying to get everyone to conform to one particular social norm, at Corgibytes, we are constantly trying to adapt our operations so that people don’t have to code switch during their experience as a candidate, and if they get hired, as an employee, too.

An Applicant-Centered Approach

We believe that people are, well… people! Each person who takes the time to fill out our application has given to us a part of their life that they’ll never get back. We honor that contribution and recognize that behind the words they typed or checkboxes they checked, there is a human being with thoughts and emotions. Each person who fills out an application gets at least two responses: one to let them know we received their application and one to let them know the outcome of their application (either we are offering them an interview or we are going with another applicant).

Andrea’s background in sales and customer experience played a big role in how we designed our process. From the very first employee, she emphasized how even though it’s tempting to defer to practices that feel quick, efficient, and “normal,” we always need to consider the perspective of the applicant in any recruiting decision we make.

Of course, as an employer, we do need to evaluate lots of people and ultimately make decisions about who we think will be the best person for a given role. As a small family-owned company, we can’t hire everyone we interview, but we can design processes where people who do interview with us get treated with dignity, respect, and walk away feeling like their time was well spent. We approach our process with humility and keep in mind that applicants are interviewing us just as much as we’re interviewing them.

Over the years, we’ve gotten feedback from candidates that this approach has made a huge difference. For example, one candidate (who we ended up having to turn away), wrote an article titled “The Best of Remote Unicorn Companies of 2019” where she shared her experience of going through the process:

”I recently had the opportunity to meet with the leadership team while interviewing for a role with Corgibytes. Every single step of the way was an enjoyable experience. I couldn’t even be upset when they went with someone else for the position, because they were all so darn nice and empathetic!

The company checked off every single piece of criteria and more! I was heartbroken when I wasn’t offered the position, but the entire experience could only be described as magical. The ultimate Unicorn.”

When we read this, we were so grateful. This is the experience we try to make sure every candidate walks away with. While we’ll make mistakes and there’s always room for improvement (more on that below) this is evidence that our applicant-centered approach works — both for us and for the people who spend their precious time interviewing with us.

Grounding Our Process in Core Values

One of the ways that we keep our process human-centered is by anchoring decisions to our core values. It brings me a lot of joy to talk to people who are considering working for us. I get to gush about how much I love working here and tell them that the culture on the inside of Corgibytes is consistent with what you see on the website. That’s largely because of our core values. They are alive in our operations and are regularly invoked when we’re consulting with our clients, creating something new, or making tough decisions.

Act with Empathy - We put ourselves in the candidate’s shoes. How would we want to be treated during this vulnerable process?
Adopt a Growth Mindset - We put aside personal biases (as best as we can) throughout the process by having multiple interviewers involved. A combination of scores allows us to grow our team as ethically as possible.
Calm the Chaos - We don’t put pressure on our candidates to make a choice right away. We understand that things like benefits packages and compensation will need to be thoughtfully considered, often with input from other people, before making a decision.
Choose Candor - If/when we need to pass on a candidate, we give them the opportunity to ask for feedback. We’ll provide kind but supportive details about what made us choose to go with a different candidate.
Communication is Just as Important as Code - Thorough and thoughtful communication throughout the entire process is critical to us. We want our candidates to always know where they stand and what to expect next.
Craft in Context - Our application and interview process doesn’t contain a lot of extra fluff. We ask questions with meaning that directly let us know how good of a fit you’ll be for the job. We don’t care what school you went to or how many words per minute you can type.

Making Mistakes

While we certainly strive to meet the ideals and standards we’ve set for ourselves, we’re not perfect. We definitely make mistakes and know there are ways that we can do better. For example, our team isn’t as diverse as we want it to be. The time to go through the process is longer than we’d like. As a small company, synching the timing of making a sale and bringing someone on board is a huge challenge. We recently read some research that a part of our process that was highly recommended as an equitable practice has since been shown to have the opposite effect than what we intended.

There have been communication glitches, too, even though we put so much thought and care into it. We’ve accidentally ghosted a few candidates because we thought we sent an email but we actually didn’t. We’ve also on occasion mistakenly used the wrong name, spelling, or pronoun, for someone, too. We know this can hit at the core of someone’s identity and recognize how much this can hurt, even if we didn’t mean to make the mistake and even if we try to make it better. Impact matters more than intention.

In all of these cases, we do our best to take accountability and learn from our mistakes. We admit when our actions weren’t aligned with our values and try our best to make it right. We’re proactive in looking at industry research and scouting for potential problems. As a team, we support each other. We help each other find grace when we fail so that we can learn and grow and do better next time. We also really value feedback. When we’ve messed up, we’ve really appreciated it when candidates have let us know. Feedback is a gift, even when it’s hard to hear sometimes.

Our sincere hope is that anyone who applies to Corgibytes feels seen and appreciated. We don’t want to be another company that ghosts you as soon as you reach out. That’s not good for you, for us, or for the industry as a whole. Here’s to doing things a little differently in the hopes that it becomes the norm!

Conflict as a Tool for Healthy Team Growth

Thu, 16 Dec 2021 00:00:00 +0000

It’s 3 a.m. You’re wide awake. You’re having a “conversation” in your mind with a colleague, a boss, a client. Again. The same one. Seemingly on repeat for the past few days, weeks, or months even.

Barring some intractable factors, it appears it’s time to transition that hard conversation from inner monologue to outer dialogue.

The thought of it is potentially making your heart race, your palms sweaty, your mouth dry, and even causing you to be a little nauseated.

When asked, most will say that they don’t like conflict. And that’s fair. I, myself, do prefer my conflict in works of fiction.

But all that internal turmoil, the time spent spinning, the miscommunications due to avoidance is not only unhealthy, it also frequently builds up and makes the situation worse.

Yes, conflict can be intimidating. And with a mutually agreed-upon approach, it is possible to elevate it to a healthy and productive conversation. As the Corgibytes Choose Candor value states:

“Healthy conflict in a psychologically-safe environment is a critical component of a high-functioning team. We hold each other accountable and strengthen our work by challenging ideas respectfully and directly. Speak up and share your point of view, even when it’s hard.”

Although it is easier said than done, there are a few concepts that we can apply to both help prepare ourselves as individuals and create that safer collaborative space.

Root the exchange in respect

Depending on the parties involved, it may be necessary to explicitly agree that the tone, wording, and sentiment will remain respectful. There is no room for passive-aggressive statements here. It’s essential to hold ourselves accountable to providing factual points to augment the discussion and help provide an additional point of view. And, also, to remember that there is a difference between sharing information and lashing out.

Accept that it’s going to get uncomfortable

One of the most difficult aspects of accepting the discomfort is that it might trigger memories or sensations of past unhealthy conflicts. Aside from setting up ground rules such as the above, aiming for high levels of self-awareness and practicing self-talk can illuminate these potential filters and help recognize, on a deeper level, that this situation is different. With practice and experiencing an increase in healthy conflict interactions, it does become easier to manage the discomfort. Even if it never fully goes away.

Keep in mind that it’s not about “winning”

Assuming good intentions from the parties involved, we should all be on the same side. As in we all want what is best for the organization, the project, etc. Feeling passionately about certain topics usually indicates a deeper level of caring. If we didn’t care, we wouldn’t be passionate. And we may have valid reasons to believe that if certain things happen/don’t happen, it could be catastrophic. In those instances, try as much as possible to present objective data or proof as opposed to more subjective arguments.

And in case of impasse, explore the possibility of a “third option” outside of the ones being considered. Often, collaborating away from two opposing points of view to extract an entirely different solution yields a new, even better, way out.

Take a moment to breathe and reflect

Sometimes, we just need to hit that virtual pause button. It’s perfectly acceptable to request time before agreeing to a solution to the issue. I’ve frequently asked for it myself to create emotional distance from the discussion. I want to ensure that my thoughts are rooted in facts rather than influenced by how I’m feeling at the moment. Be upfront about needing to let the points of view sink in.

Choose which topics to address

Sometimes, when we’ve been avoiding conflict for so long, there is much to be discussed. Refrain from an “emotional dumping” in an attempt to solve all the things all at once. Not everything needs to be said and discussed “right now”. Select the most pressing issues that impact the business, team, morale the most and address these first.

And now… lean in

In my experience, the more I choose to lean into the discomfort and see these difficult conversations through, the more I become used to them. And the more I see the positive results of these, the more I have them.

Listen with an open mind. Leave lots of room for each other to talk. Self-regulate the emotions. And go for it.

Yes, there will be times when elements won’t be resolved successfully. And there will be times when acceptance will be necessary.

Despite that, I’ve found that for the most part, these hard conversations do generate some form of progress. Even though I don’t particularly enjoy having them, it does beat the alternative of having these on repeat, in my mind, at 3 a.m.