TeXtify

LaTeX can be hard. Create beautiful and professional notes, research papers or assignments with just your voice.

Comment

Inspiration

We are both engineering students, and spend countless hours writing out math, whether it be by hand or by tediously setting up a LaTeX environment and slowly learning how to use it. Now, anyone regardless of how neat their handwriting is, or how much they know about LaTeX, can create beautiful, professional math documents. All you need is a microphone and your voice!

What it does

TeXtify listens to your voice through your microphone, and converts it into LaTeX code, which is then used to create a customizable, beautiful LaTeX document right in your browser. All you have to do is say your equation out loud and we handle the rest!

How we built it

We built TeXtify using a React (TypeScript) frontend. We used the Google Cloud speech-to-text API to transcribe the user's voice from the microphone into text. We then pass that data to a Python FastAPI backend, which provides the data along with a very specific prompt to Cohere's generate API to get back the LaTeX code, which is provided back to the frontend. Both the client and the server are hosted in a Google Cloud Run serverless environment, which allows them to scale vertically independent of each other based on user demand.

Challenges we ran into

Designing the prompt so that we got clean LaTeX code with no extras and as few mistakes as possible was quite difficult. The LLM had a tendency to treat its response as a conversation, adding a bunch of text to provide context, which while appreciated, was not required for us. State management was also a pretty big challenge in React (as usual). It was especially difficult for us because the editing done in the Editor pane had to immediately update on the Document pane, leading to some complex state interactions.

Accomplishments that we're proud of

We are very proud of how we were able to turn TeXtify into a project that we would use for ourselves in such a short amount of time. It will prove invaluable during exams when the wrist fatigue from writing continuously will kick in.

What we learned

We learned a lot about how LLMs can and will be used in software applications. They can be used for so many things, not just chat apps.

What's next for TeXtify

There is a lot we wish to do for TeXtify in the future. We think that this could change the way we write notes and assignments for university, and we are very excited to add many new features, such as multiple documents with saving your progress (the backend is actually done for this we just didn't have time for the frontend), and a full speech-to-text interface, not just for math but for the titles and headers too. We would also like to better tune the LLM to understand and generate LaTeX code.

Share this project:

Updates