Inspiration
Vector databases are a hot new topic - and their querying abilities allow for magnitudes faster improvements over traditional databases. However, generally speaking, even if the data is encrypted, the embeddings are a vulnerable spot in the database's security. We decided for sensitive medical data that we should try to make a database with better security than existing solutions by not only encrypting sensitive images, but the embeddings as well while sacrificing as little speed as possible.
What it does
We implement signin/signup using Auth0. We focus on Melanoma skin cancer in order to maximize our model's effectiveness by focusing on a small domain for both of our models. Not only that, we currently have API endpoints for both uploading and querying images of skin cancer. Images that are uploaded can be classified by our model. Not only that, we can query for similar images to the ones users upload in order to find similar patients with potentially useful diagnosis.
Not only that, our frontend was not sophisticated enough to effectively show the information from our backend. While queries can be accomplished through our API endpoints, sadly we cannot display them on our website. Not only that, there was insufficient training time for our fine tuned model to be as effective as we liked it to be.
How we built it
We used Auth0 for signin/signup. We also fine tuned models in order to classify images, as well as generate embeddings for our vector database. The vector database solution we decided to use was MongoDB, where we implement image querying. We also used NaCL to encrypt our images, which was one of our goals while we were unable to encrypt our embeddings.
We used flask and React to build our website.
Challenges we ran into
We were unable to transform the encrypted vectors into embeddings for MongoDB. In the future, we should have used a more sophisticated scheme in order to obfuscate our embeddings better. Not only that, our inexperience in building large scale applications showed in this project as both scope creep and inexperience began to catch up with us.
Despite our challenges, we learned a lot from this, as well as having many places to grow and improve.
Log in or sign up for Devpost to join the conversation.