HireHack

Where Skills Meet Opportunity

Inspiration

It's no secret that hiring in the tech industry is extremely tight right now. In a time where hundreds of applications to land a single interview is considered standard, the stakes for performing well in them are very high. Unfortunately, many applicants have limited experience with interviewing, and -like most people- they are unable to recognize their own flaws while presenting. Inspired by this paper*, this tool aims to help applicants improve both what they say and how they say it.

*chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://roc-hci.com/wp-content/uploads/2018/06/naim_et_al_FG2015_final_version.pdf

What it does

HireHack is a web extension that is used to provide real-time feedback to interviewees based on their facial expressions; raw language, including filler words; and actual speech, including tone & speed. It runs a separate data model for each of the three main parameters on a Flask server, which is connected the the actual JavaScript file that the HireHack extension is produced by.

How we built it

Our project consists of several components, all linked together:

A JavaScript based chrome extension that is recording both audio and video, converting the audio into text and saving this as a transcript
A flask server that is running the three models (the facial emotion recognition model, voice analysis model, and lexical analysis model) and generating a combined output for the three models
A feed forward neural network that takes the aggregated output from the three models to predict the improvements to be made
an LLM that converts the raw numbers and labels of the output into human understandable text

With the help of these four components, the web extension is able to abstract the entire "recommendation generation" part and provide real-time information on how the interviewee can improve.

Challenges we ran into

--> Deploying the Flask server (the "." to reference loops and the 30 minute generation + deployment time) --> caching (Thank you, professor Gustavo) --> Using LLM APIs at a low cost --> Cloud Storage in general

Accomplishments that we're proud of

--> We truly went out of our comfort zone for this one! --> Deploying the Flask server on Render when we had no practical experience with either of them (Brutal hours of debugging and questions from Mentor later) --> caching --> fixed! --> We built a fully working Chrome extension from scratch --> Integrating multiple modules together

What we learned

We enhanced our JavaScript skills, learnt how to train and build data models, learnt how to create and connect to a Flask server, understood the challenges of accessing LLM APIs at low costs, and learnt a lot about cloud-storage.

Additionally, as four sophomores limited to theoretical experience with cloud-based development, building/running models, and linking Javascript files to servers and LLM APIs, we experienced a steep learning curve during our project. We learnt that while being ambitious with our goals is important for to learn and develop things of value, being overly ambitious can cost us too.

What's next for HireHack

1) Figuring out exactly what went wrong with our deployed version 2) Working with that information to create a fully functional HireHack! 2) Expanding our datasets to enhance our models for each of the 3 broad parameters, and considering other parameters in an equally important light