\LeTEXT

Inspiration

As CS Students, math enthusiasts and amateur researches, we know the struggle and the beauty of watching code become artistic, organized writing. Yet, it is so difficult to become comfortable without extensive experience. We hope to enable anyone to quickly form perfect documents, quickly and intuitively.

What it does

The crux of our application is a Fine-Tuned LLM Copilot, OCR for handwritten math, and custom commands integrated with a blazingly fast native LaTeX IDE and Renderer in browser.

We have built in commands for common things like logical-and, logical-or, and even Demorgans Law so that your paper or proof shouldn’t be hindered by a curly bracket. The main part of our product is an AI model using Together.ai with CodeLlama-2 trained on a large data set of 450 common prompts in Discrete Math and research papers. A prompt like “create a bullet point for my education section on my resume at the Stanford” would spit a perfectly formatted block of latex in our proprietary IDE. Or “perform DeMorgans law on Set A and Set B”. This is chips and guac for us. Have messy handwritten math? No problem at all!

How we built it

The backend was built using a TexLive renderer in a Docker container that is run to consistently render the Tex Document continuously so you can see your changes “live”. The frontend was built with Typescript + React with Vite, and the backend with FastAPI, Firebase Auth for authentication, and Postgres for database storage so that users can save to the cloud in the future. This was used in tandem with Monaco Editor from Microsoft for all the logic we embedded to deal with the autocomplete and code snippet replacement commands. For the ML Model we trained it using Together.ai for a fine-tuned CodeLlama-2. For OCR, we leveraged MathPix APIs.

Challenges we ran into

We found no shortage of challenges. Our biggest issue was giving the ML model permission to write into the IDE as the package configurations coupled with the extensive setup for the Tex Renderer to take this input. We decided to approach it using custom commands (such as /ai) to give the user command over when they want to get help. It was quite the challenge. Of course, building the container and integrating it with the backend was no small feat either. We put these different puzzle pieces together and realized there was more color to the bigger picture than we anticipated.

Accomplishments that we're proud of

We are most proud of creating an integrated IDE. None of us have experience making developer tools so we were very proud with how we learned the tech to power our application. The key thing is in Monaco Editor Latex was not a supported language compatible with the IDE. As such we had to bruteforce produce mappings of the key words like \textbf{} and link smart highlighting with the intellisense that runs Monaco Editor to get a fully functioning IDE tailored to our use cases. As an extension we are especially proud of our Latex CoPilot. Utilizing Together to integrate a smart AI code generator that intakes English semantic was an enriching experience to accomplish, especially building the model from the ground up and linking it to the IDE.

What we learned

We see ourselves enhancing the ML model to implement more tailored Tex code and even make suggestions based on what the user wants. Furthermore, we see ourselves implementing our own syntactical language that overlaps Tex that makes it more user friendly maintaining a consistent lexical structure throughout, as to interpreting English input and immediately translating it into Tex. Finally, Nonetheless, we enjoyed this project and were able to see the fruit of our work taking our old Discrete Math work and research papers being written in this app of ours.

What's next for \LeTEXT

We see ourselves enhancing the ML model to implement more tailored Tex code and even make suggestions based on what the user wants. Furthermore, we see ourselves implementing our own syntactical language that overlaps Tex that makes it more user friendly maintaining a consistent lexical structure throughout, as to interpreting English input and immediately translating it into Tex. We also intend to add cloud storage for users. Nonetheless, we enjoyed this project and were able to see the fruit of our work taking our old Discrete Math work and research papers being written in this app of ours.