Win-terview

Recording Page
Dashboard Page
Win-terview

Inspiration

As students applying for internships and jobs, one of the biggest challenges we face was acing interviews. Besides the answers to interview questions, one big factor on whether you will pass the interview are the non-verbal cues and how well you can deliver your speech. This was something so important yet not formally taught in school. Hence we decided to build a web application to help students improve their interview skills.

What it does

Win-Terview helps students practice and improve their interview skills by providing them mock interviews, and live feedback on their answers based on non-verbal cues and speech delivery.

How we built it

The facial recognition is done by using Google Cloud Platform Vision API. The voice recognition is done by using Google Cloud Platform Speech-to-text API The video recording is stored on google cloud storage, and then accessed by our python programs that implemented the Google APIs.

For front-end stack, we used HTML, CSS, Javascript, and jQuery to build the website. From the front-end, it used RestAPI to upload the video data and retrieve analysis results back to display the interview score.

Challenges we ran into

We wanted to analyze tone of voice in the text but we did not have enough time to scope that out.
Insufficient time to do proper research on the specific verbal and non-verbal cues which gives interviewers a good impression. This made our product not as holistic as we wanted it to be.

Accomplishments that we're proud of

-Using Google Cloud’s Vision API and Speech-to-Text tool, we managed to build a multi-dimensional tool that could read facial cues and evaluate speech delivery, and offer immediate feedback to users too. We built a product which we would all use and that’s awesome! -Users are also able to playback the video of themselves answering the interview questions, and get relevant tips for specific areas which they need improvement in.

What we learned

-We learnt that Google Cloud actually provides a wide variety of tools and APIs which could be useful in a wide variety of projects

What's next for Win-terview

Gather more non-verbal cues that makes a good interview response
Add those variables into our facial recognition and voice recognition software, to provide more holistic feedback for our users

Built With

css
google-cloud
html
javascript
python
speech-to-text
vision

Submitted to

BostonHacks 2019

Created by

I worded on the back-end. I extracted .wav audio files from .mp4 video files. Then I use the Google Cloud Platform Speech-to-text API to analyse the audio file. The text file is then analysed together with the audio file to score the interview based on stuttering, use of filler words, and speaking speed.

Sing Yu Yeung
Mainly worked on front-end and web development. I worked with HTML, CSS, and jQuery to build a website.
Created a dashboard page to show historical data of the user’s interview progress, ability to play and record video on front-end.

Hanbin Chang
I developed the idea and researched on the non-verbal cues and metrics we could use to determine whether a interview response was good or not. Designed the logo and the presentation too

Gwen Sim
I worked on the back end. I used Google Cloud Vision API in Python to detect facial expression in the pictures extracted from two interview videos.
Created a mechanism to compare the good/bad interview performance by assigning a weighted score to the detected facial expression.

Dylan Kuo
Private user