Inspiration
We were inspired to bridge the virtual world to the user's physiology. We began with the goal to create somekind of wearable that acted as a 'smart skin' or exoskeleton that augmented the wearer's awareness of their body. Making the invisible world visible.
We assessed the tools available, (devkits, software, hardware) and ideated on key areas of interest to apply AI to enhance human behaviour. Tapping into medical knowledge and also direct, real-time feedback to make the knowledge applicable to the user's immediate circumstance.
What it does
Our biofeedback smart sleeve uses a combination of tech: 1) Computer Vision feeds the system data about joint position. 2) EMG (electromyography), measures electrical activity in muscles. 3) Haptic feedback guides the wearer to adjust their position into 'trueform' position.
This enables us to map the wearer's technique during physical activity, whether that is: 1) Job function, like picking up boxes, or performing any physical work tasks. (Relevant for workplace health and safety) 2) Fitness, such as mapping technique while lifting weights. 3) Sports, such as refining techniques to optimize performance and mastery. 4) Arts, such as learning choreography in a dance studio. 5) Everyday actions like walking, sitting down, clmbing a ladder etc.
Futurescope: By using this data as an input to an LLM trained on medical information and physiology of wearer, recommendations would be made to align movement and train new habits to avoid injury.
In the case of an injured patient, medical professionals could specify movements to do/avoid as part of rehabilitation.
Our celebrity DLC, (Downloadable Content), would add additional references for technique, so users could compare their movement with sports stars they aspire to play like: "Shoot hoops like Kobe Bryant", "Kick goals like Messi".
How we built it
We separated the hack into different sections, The hardware electronics, the interface for visualization and the wearable components.
Electronics We connected the haptics to the OpenBCI Cyton board and streamed it with the EEG data to unity. We then used the Qualcom video sensor to take visual input to run AI analysis to track exercise movements and detect sub-optimal movements.
Visual Interface We designed a mobile app for users to visualize that data and get recommendations on what to do for each exercise as well as a plan.
Challenges we ran into
Getting data off of Qualcomms RD3 Controller Normalizing EMG data after long periods of time Customizing functionality of the Cyton EEGs pins Running a pose recognition model on both RD3 and our computer Printing 3D designs
Accomplishments that we're proud of
We were able to track fine detailed EEG data from muscle movements. We used openCV to visualize precise target locations on the body in real time. We visualized the data analytics in Unity We created a mesh 3D model for our weasel and printed various prototypes We created a working circuit that used the EEG data to trigger mosfet, haptics and connected it on the wearable. We designed a mobile app to go hand-in-hand with our product. We made the data available through a UDP for XR visualization and AI analytics on different platforms in the future. We created a dashboard for realtime analytics.
What we learned
We reverse engineered a 3D printed mesh as a fabric, (studying the NASA design for their interlocking cloth). We modelled our own version and 3D printed it and compared it's movement potential to the original reference and are continuing to refine the design to enable more flexibility for movement in both concave & convex positions.
What's next for TrueForm
We will be refining the prototype and surveying different industries, (fitness, medicine, arts etc), to see if there are other data points of biofeedback that would be useful to add to the smart sleeve.
We would also like to increase the consistency of signal from the sleeve, so will be exploring how to make the electronic connections stable, while still allowing for maximum movement by the wearer.
We would like to land a research deal with a professional sports team to study their movement in their craft and playtest our data collection methods.
We intend to start a membership group so a community can grow around the product.
Log in or sign up for Devpost to join the conversation.