The term “Deepfake” is referred to a deep learning based technique that swaps the face of a person with another face in an image.

It is easier than ever to create deepfakes of anyone using the tools available online. Deepfakes can be used by people to generate fake news, hoaxes, blackmailing, financial fraud, and many more malicious activities.

Deepfake Shield is a tool that uses deep-learning to detect deepfakes in an image. The diagram below summarises our project. Feel free to try out the web-app - https://deepfake-shield.herokuapp.com/

Our favourite way to use mlrun has been the # mlrun: start-code and # mlrun: end-code. The ease of use in terms of tracking experiments helped us progress rapidly from experimentation to training and deployment, all without the hassle of trying to keep logs manually.

When preprocessing the data, mlrun.artifacts.PlotArtifacts helped us visualise a bias in the dataset. We found that the number of real images is much lower than that of the number of fake images. This was fixed by inflating the number of real faces using the FFHQ dataset

Finding suitable training hyperparameters manually can be quite tedious. We automated this process using mlrun.new_task().with_hyper_params(grid_params, selector="min.loss"), thus making the process of finding hyperparameters a lot less painful.

The pipeline that we built comprises of 2 different models:

• We use a pretrained BlazeFace model (which can be retrained if needed) for extracting faces from images.
• We trained a customized implementation of EfficientNet for classsifying the extracted faces accordingly.

The model was trained using the ideal hyperparameters found using grid-search with mlrun. The training and evaluation logs were also tracked using mlrun

The webapp can be run locally with the following steps:

1. Clone the repo and navigate into the folder

git clone https://github.com/Mainakdeb/deepfake-shield.git

cd deepfake-shield

2. Install requirements (venv recommended)

pip install -r requirements.txt

3. Run webapp on localhost

python3 app.py

• Data exploration: NBViewer, Github
• Preprocessing + Hyperparameter search: NBViewer, Github
• Training + Evaluation: NBViewer Github
• Dataset: We've used a modified version of the deepfake-detection-challenge dataset.