DevShack

Inspiration

Programming can get a little hectic when you're honed in on solving a complex problem, and you have to pull yourself back to remember to use version control. While using Git is undeniably crucial, crafting meaningful commit messages becomes a hassle when the code base is evolving rapidly.

What it does

To tackle this, we devised a straightforward solution, feeding the code changes into a large language model to automate the creation of commit messages. When we saw how effective our implementation was, we actually started using it during CruzHacks as we increased the scope of our project. Taking it a step further, we implemented a parser that utilizes language models to generate inline documentation, effectively eliminating the tedious aspects of future programming assignments for the rest of our academic career.

How we built it

Embarking on CruzHack, our team decided to spice things up by learning and making our project in a new programming language. Our first milestone for the hackathon was actually learning Rust. We arrived early to claim our table, put our heads down, and learned Rust for the entire first night. Once the sun rose at 6AM Satuday morning, we were pumped with caffeine and ready to actually start putting together our program.

We implemented a simple command handler with plans on expanding our program beyond the original scope. To get commit messages written for us, we called git to produce code changes that is then piped into an API request to send to the Mistral 7B LLM hosted on Cloudflare. With a bit of prompt engineering, Mistral 7B was able to produce a 128 character messages that fit the conventional standard of commit messages, that we then pipe into a git commit -m call to actually commit the code! That way users didn't have to manually copy and paste the generated message when performing a commit. We took great care to streamline this process, requiring almost zero effort on the user end so that they can just focus on the programming. Once we saw how effective this implementation was, we took it a step further by considering what else was tedious during the development process. We quickly agreed that it was writing documentation for our functions. Our team actually consisted of members that had a background in web development so we actually wrote the code to write JSDocs for our JavaScript and TypeScript programs.

Challenges we ran into

The first problem we ran into was command handling. Our program had multiple features, and implementing a command handler ensure well organized and modular code. We spent the entire first night implementing a sound and scalable command handler which formed a strong foundation for the rest of the project. The next big challenge was writing effective prompts for the language models. Asking these models to "write a meaningful description of these code changes" did not consistently generate meaningful commit messages. 8 out of 10 times the messages were subpar and worse then anything we could write with minimal effort. So we put on our prompt engineer helmets and just tested several different models with different prompts to write commit messages. We found that the Mistral 7b model hosted by CloudFlare generated consistent good results. The biggest challenge we ran into was when we proceeded to implement the feature to write inline JSDocs for our JavaScript and TypeScript programs. We needed to figure out how to parse the files to locate the functions to keep track of their positions so that when we received the JSDocs from the language models we didn't overwrite the functions when writing the comments inline. We ended up using a library called Treesitter to parse JavaScript and TypeScript, but that meant this feature only supported those two languages.

Accomplishments that we're proud of & what we learned

Our goal for this hackathon was to learn and use Rust. Our team consisted of members who came from a web development background (hence the language models only writing JSDocs), so we actually learned Rust as we were working on our project. We're extremely proud of this accomplishment and getting a working project was just icing on the cake.

What's next for DevShack

We developed this project with full intentions to use it everyday for our programming assignments. This means continuous tweaks and changes to the program until we're happy with how it functions. Some of us may fork it to make specific changes for their workflow in the future.

Built With

• cloudflare
• openai
• rust
• together.xyz
• treesitter