Manimify Math

Real-time generates and animates your own custom math visualizations using computer vision and generative AI.

Comment

Manimify Math: Bringing Math to Life with Computer Vision & Generative AI

Inspiration 💫

Inspired by the visually mesmerizing animations of 3blue1brown, powered by AI, our goal was to transform traditional math problems into dynamic visual experiences to make learning like storytelling, both engaging and intuitive for students and all. We wanted to take the problem and bring it to life.

What it does 🌠

Here is what our application does! :

  • Uses your smartphone’s camera to snap a picture of a math problem
  • CV via the Mathpix API to extract precise LaTeX strings from images.
  • Feds it to OpenAI API to craft custom Manim script templates tailored to your specific math problem
  • Utilizing the Manim open source library to produce high-quality, animated videos that bring abstract concepts like the 3D volume of a disk in calculus to life, making it easier to visualize from 2D paper.

How we built it

Here are the technologies we used: 💻

  • Mathpix: Our app starts by using a camera connected to your phone from this API and uses computer vision to accurately convert images of math problems into LaTeX code.
  • Script Generation: The extracted LaTeX is fed into the OpenAI API, which generates dynamic and visually appealing Manim scripts which we adjust around the angle and 3D view.
  • Render with Manim: The custom scripts are executed using Manim, an open source animation rendering library, for high-definition animations in hopes of clarifying even the most challenging concepts.
  • Integration: Synchronous API calls to make it seem seamless.

Challenges we ran into 😳

OMG. Connecting everything together was not the easiest. We started out by one of us doing the Manim template and the other creating the scanner to LaTex. Then we had to connect it with OpenAI API that automatically edits the script template. These three were consuming in itself, but connecting them all together took the most time. So much. We had 2 API calls running simultaneously, calling one another form different scripts, and sometimes it wouldn't run correctly. We had to transfer script-specific variables from one script to another with accuracy, or else the video would not render correctly.

Accomplishments that we're proud of 🤗

We successfully merged CV, AI, and high-quality animation into a single, cohesive application. Our tool transforms abstract math problems into visual narratives, helping students grasp complex concepts with ease. We got to use our favorite YouTuber's library and learn more about how to use it to enhance learning.

What we learned 🤩

How to use Manim!! We had never used the library before this or Mathpix API either. We learned how to use both, but Manim is some pretty cool stuff.

What's next for Manimify Math ☄️

If we had more time we would have an enhanced user interface since it is bare bones right now. We plan to expand our tool to support a wider range of math topics and complex problems. Future updates will introduce interactive animations, allowing users to manipulate the parameters. Also having it run by itself instead of locally.

This was super fun and exciting to make, we hope to enhance learning for all!

Built With

  • git
  • manim
  • mathpix
  • openai
Share this project:

Updates