LaTeX 2 Speech

A web app that reads any LaTeX equation out loud.

Comment

Inspiration

Students around the world use the document preparation software LaTeX. Particularly, it's popular for converting plaintext into well formatted equations. Everyone has their own way of learning, and we noticed that LaTeX doesn't have auditory support for equations! This can suck for students who like to learn this way, or students with visual accessibility needs. Professors won't read out all equations in lectures, so it would be nice for something else to fill in those missing equations. We built this app to help students learn better.

What it does

Go to text2speech-la.tech (latex -> la.tech :P ). The user uploads an image file of (.png, .jpeg, .jpg) format or a .tex latex file. The web application finds any equations in the file and returns an audio file of a TTS voice reading out the equation.

How we built it

The application uses ReactJS for the frontend and we connected it to the google cloud server with axios. If the file is an image file, the server sends it to an OCR in the backend which converts equations in the file to a LaTeX string and sends it back to the server. For all files, the LaTeX string is then changed to English by another backend application, and finally the English is sent to a TTS API which sends an audio file back to the frontend. For dev ops, we used google cloud and initially tried to use GitHub actions as it had better integrations with the code we were writing. But as we started to use more google cloud services (eg. cloud run, cloud artifact, cloud firebase hosting), we decided to move CI/CD into cloud build. It had a high learning curve, but it was worth it in the end. We also implemented docker layer caching in cloud build triggers to reduce build times from 15 minutes to 3 minutes.

Challenges we ran into

Connecting frontend to the server gave us a few problems, and there were some issues with auto-deployment. We brainstormed a bunch of great ideas to add to our app, but they weren't realistically achievable in 24 hours. For the backend, everything was new, and it had a steep learning curve. Specifically, for DevOps, the feedback loop was very long, so developing the pipelines took a lot of patience.

Accomplishments that we're proud of

For the frontend team, we learned how to send and receive data from the server. Even though we couldn't get them completely connected, there was proof of concept. For the backend team, getting the dev ops to work together.

What we learned

We should start small with the core functionality of the app before worrying about more extra ideas.

What's next for LaTeX 2 Speech

Handling multiple equations, perhaps so that a list of audio files is generated.

Built With

Share this project:

Updates