Inspiration
This powerful application will allow our users to smoothly navigate the web in a way they feel most comfortable.
What it does
Our project uses voice recognition and eye movement tracking in order to execute web browser commands without having to lay a finger on your computer.
How we built it
We created a Google Chrome extension and used a neural network database called TensorFlow. We integrated Selenium to execute commands via voice recognition. Also, with the help of WebGazer, we were able to track eye movement to automate scrolling within the Chrome browser.
Challenges we ran into
If you are on MacOS and attempting to configure your chrome webdriver executable you do NOT need to add .exe to your file name. You receive the error "please add chromewebdriver to your PATH" if you make this error.
If you wish to recreate this project I highly recommend you do so with PyCharm. It will help your configure your virtual environment. It will also allow you to install Python packages within the editor (preferences>Project[Name]>Project Interpreter>"+").
Accomplishments that we're proud of
We are proud of the amount of work we were able to code/work through in the short amount of time we had. Even though we had to pivot our idea, it all worked out at the end.
What we learned
- How to make good use of python packages to implement voice recognition/eye-movement tracking
- How to make a Google Chrome extension
- How to train a neural network
What's next for Lava
We plan on expanding on our Google Chrome extension and further training the neural network to improve accuracy.
Built With
- chrome-webdriver
- javascript
- pyaudio
- python-3.6
- selenium
- speechrecognition
- webgazer
Log in or sign up for Devpost to join the conversation.