Inspiration
DerWaldi wanted to read into his scientific papers on the go. Having your hands free would be a major benefit. So he tried listening to them. However, mathematical equations did not work at all with the pdf reader of his choice. Pitching the idea to us, we remembered lectures on accessibility for blind people and the lack of a good reader for scientific papers, especially mathemathical terms. We hope ReadMyScience can not only help DemWaldi, but also some other people as well.
What it does
ReadMyScience reads your scientific papers out loud, including the math terms.
How we built it
We use a python backend, tying everything together. Using tensorflow we tained the yolo object detection net to detect mathematical terms. We used the MathPix WebAPI to get Latex code from the equations. After that we parsed the latex code with python in order to make it human readable/listenable. Finally we used Google TTS to read it out loud.
Challenges we ran into
TextBoxes in PDFs are hard to extract and hard to get into the correct order. PDFs are only goof for presenting, not for automatic data extraction. Determine how to read aloud mathematical terms.
Accomplishments that we're proud of
Finding and identifying the equations in pictures at all :D
What we learned
It is a hard problem with many more things TODO
What's next for ReadMyScience
Get it on the smartphone, so DerWaldi can listen to his papers :D
Log in or sign up for Devpost to join the conversation.