Croppery

Quick Note: thumbnail Image from text2image on replicate using the prompt "croppery #artstation". Weird image but gets your attention!

Inspiration

I was in awe. I discovered replicate yesterday and the amount of cool stuff (with a lot of potential for great use-cases) that's on there is amazing. I thought about what I could do with one of those cool models and cropping images with Facebook's Detic seemed like a great idea.

What it does

Detects common objects in an image, allows you to crop them instantly. Also allows you to specify a list of words (any words!)

How we built it

Used Detic for the "brains" of the project, Flask to serve the model predictions to the front-end, and React for the (non-existent) front-end.

Challenges we ran into

A lot!

First I used Heroku, hated it.
Then render, loved it.
But the model uses too much RAM for render!
Couldn't deploy as an api/Model-as-a-service (don't know how and no time to learn)
Decided to deploy locally instead.
Bugs with front-end sending images to back-end, back-end loading the model, installing dependencies (detectron2 is a nightmare!)

Accomplishments that we're proud of

it works! (sort-of)
Made in 26 hours (I discovered the hackathon sort-of late)

What we learned

A whole lot!
It has finally motivated me to start exploring ML model deployment.
A lot of firsts here:

First time I use render.com
First time I try image cropping in React
First time I use gunicorn and a bash script for deployment
Discovered a lot of great services using ML online while researching this
First time I used OBS studio to record on Linux (which is why shotwell images are blacked out!)
A lot of more minor things I can't put into words but have nonetheless made their cognitive mark on me