Few Words About Using GitHub Copilot at work

tl;dr:

• CoPilot can be really useful, and I don’t want to miss it anymore.
• Nevertheless, it’s important to keep in mind that it’s currently not the universal remedy for developer productivity.

By the end of the past year, the company I’m working for bought Github Copilot licenses for (nearly?) all developers.

I was curious if and in which way it helps me in my daily work. So, I took notes about my thoughts.

55% faster coding?

I think I use Github Copilot as most users do: As enhanced autocompletion and for harder, not so obvious solutions via the chat.

Every time I develop in my free time, I feel how much I got used to the autocompletion, as I don’t have GitHub Copilot for my personal use.

But I wouldn’t agree on GitHub’s 55% faster coding promise. GitHub Copilot is helpful for boilerplate code and can generate tests out of the context it gets from the workspace.

The hard task that takes most of the time as a developer is imho not writing boilerplate code: It is writing business logic and components in the wider context. You need to think of how the components interact with each other and if a change at one place can affect another place.

Communication between different stakeholders is also another time-consuming aspect. You get the point and I’ll keep it as short as possible because that’s not a post about the replacement of developers by AI.

Where it’s good

That’s a list of things which I find good. Where good means: It can be used, but must be used with caution as it’s not 100% right what it does. But most of the time these use-cases can really speed up development:

• Generate a first draft of a piece of code when writing a comment/documentation. This can be especially helpful for the discovery of an algorithm or an API which is not well-known.
• Generate tests for a method. Generally, ask for edge cases when looking at a piece of code.
• Explaining a piece of code which is unknown or bad to read.
• Writing documentation. Autocompletion also works for text and documentation and this can be helpful.
• Autocompletion. It doesn’t mean that you place the cursor at one place and GitHub Copilot will suggest an autocompletion which makes it seem like it read your mind.
• Although that happens quite often it can also mean that you know what you need and as soon as you start typing CoPilot will fill in the rest for you.

Where it needs to improve

• Due to the training data cut-off it uses often old APIs. When I use Spring Boot 3.x it wants to use outdated classes from Spring Boot 2.x. I couldn’t find a way to give GitHub Copilot the context of the newer framework.
• It is sometimes a hard fight to let it align with code-conventions. But that’s a point which I feel got better within the past months.
• I wish it would have a function where it checks whether the documentation above a class/method still aligns with the code after some changes were done. This would prevent outdated documentation.
• It has no plan about side-effects of the code it writes. When it suggests some code it sometimes is either bad/unperformant style or it adds some method calls to 3rd party APIs which are just not needed. So, you need to babysit it.
• When I delete lines and want to replace them with a different context it tries to convince me with suggestions from the deleted lines.
• Once I wanted to migrate a .NET API to Java. I tried to use GitHub Copilot to migrate the XML-style .NET documentation to Javadoc, but it just didn’t work. This was past year, so maybe it’s better now.

Generally, often it feels like you’re fighting with it, trying to convince it to do something in a way which is intended by me. Instead, it just does it the way GitHub Copilot thinks it is the one and only way.

Conclusion

I wrote a lot about the downsides of GitHub Copilot here, but it shouldn’t sound so negative. There’s just a long wishlist which grew bigger and bigger after I used it. My expectations were not fulfilled when I used it in the first weeks, but anyhow:

I think everyone should try to bring GitHub Copilot into their own workflow. It’s a tool that doesn’t cost that much but still brings reasonable value: Getting smarter code suggestions/completions and a chat in the IDE helps in the daily work.