BlueHands

Inspiration

We wanted to work with an AR MLM! One of our members has been doing game dev, and his keyboard is SUPER gunky. So, we suggested computer control with our hands. Out came Blue Hands!

What it does

It takes hand gestures detected by your camera, processes them, and then executes a hardware control for that hand movement. We don't exactly have it flushed out and hope to update the read me. In essence, you can control your computer with your hands! (Like a jedi)

How we built it

We used YOLOX to detect hand movement. We figured that we could call functions which control your mouse/keyboard based on what webpage you were in. For basic functionality, we decided to include only mouse movements and a few

Challenges we ran into

Many of us are new to programming. We had to learn a lot in a short amount to time. It took a while to figure out how to do it.

Accomplishments that we're proud of

The Yolox model was complex to work with. We were so proud when we got it to work, it was like magic. Made us feel like wizards.

I (PineappleJamming) had extra time, so I made a website which you can see below! It is in hopes to add the chrome extension and possibly a fourm, but we ran out of time.

What we learned

We learned a couple of new programs, and how to call functions from other files in order to execute actions. We learned how to use a machine learning model and integrate that into a program. I (PineappleJamming) learned some Next.js and how to use CloudFlare in order to create our website.

What's next for BlueHands

Hopefully, we can flesh out our website and add it as a extension. The hope is also to develop full typing capability using ASL. It could be a huge game changer for how we interact with computers!

Built With

• next.js
• python
• yolox