InvoiceImageProcessor

• 

  Home view

• 

  Take picture view

• 

  Processing view

Inspiration

This App tries to solve the challenge of 'MarginEdge' in RamHacks 2020 which was to develop a mobile application that can take and process pictures. In order to do this, we propose a flexible Android app with the following main characteristics:

First, our app allows the user take pictures of invoices/receipts to process them in order to analyse the procedence of the receipt (store name), date, total cost and number of items bought. We use opencv and Machine learning models to process the picttures. Opencv is used for preprocessing. The ML models helps us to classify the image content and also extract all the text information.

Second, all the information processed is saved in Firebase (Firebasecloudstorage for the images and FirebaseDataBase for extracted information). We also maintain a local database (in device) using sqlite.

Third, the app provides a home screen that presents all the saved information as a list of items. This way you can keep track of all previous invoices/receipts or whatever image processed.

The main challenges we faced in this project were not only the need to propose a flexible camera application that uses machine learning models, but also learning for the first time the use of the ML models and opencv integration for android.

Specifically extracting the text from the images has been proved to be a difficult task. We used already ML models which were not trained for extracting information from invoices/receipts exclusively. With more time we would like to collect our own dataset based so our ML models would be more efficient.

Built With

• android
• firebase
• java
• mlkit
• opencv
• sqlite