Inspiration
Stroke is a leading cause of disability, affecting over 878,000 Canadian adults as of 2017–18. It results from a blocked (ischemic) or ruptured (hemorrhagic) brain blood vessel, with ischemic strokes being more common than hemorrhagic ones. While stroke risk increases significantly after age 55, one-quarter of those affected are under 65. A stroke event often leads to upper extremity impairments, affecting about 80% of survivors and limiting their independence in daily activities. While traditional rehabilitation methods like physical and occupational therapy are standard, they face challenges such as limited access, high costs, and low patient engagement, especially in home-based settings.
VR training has recently emerged as a promising stroke rehabilitation option, and when combined with task oriented exercises, can significantly improve stroke recovery. Studies show that VR enhances upper extremity function, balance, mobility, and walking ability in both hospital and home settings. By offering targeted, engaging, task-specific exercises, VR based therapy promotes neuroplasticity mechanisms and motor learning, making it a promising tool for improving adherence to stroke rehabilitation.
Other emerging and effective interventions for people with stroke are High-Intensity Interval Training (HIIT) and Motor Imagery (MI). HIIT is a time-efficient and enjoyable intervention that has the potential to outperform conventional neurological rehabilitation. It consists of short (under 1 min) or long (1–5 min) high-intensity exercise, divided with recovery periods. HIIT is safe for stroke patients without cardiovascular conditions and enhances cardiovascular, muscular, and cognitive function by boosting neuroplasticity, cerebral blood flow, CO₂ consumption, and neuronal activity. MI is a cognitive strategy involving repeated mental rehearsal of movements without physical execution. It activates brain areas responsible for motor planning and enhances motor learning and functional performance in daily tasks. MI of locomotor skills can stimulate internal representations of specific movements like reaching and can be practiced from either a third-person perspective (imagining someone else moving) or a first-person perspective (imagining oneself performing the movement).
A common element among these methods is the critical role of repetition in treatment effectiveness. And research has consistently shown that repetition is essential for learning. However, no existing intervention program currently delivers the combined benefits of these therapies in a single, affordable, and user-friendly setup for stroke patients. Our idea emerged from the need to provide comprehensive care that targets multiple aspects of human functionality through an interactive game, designed for both clinical. This project serves as a proof of concept to evaluate whether such a game can achieve high levels of reliability and usability, while also exploring the potential of integrating multiple effective interventions into one powerful solution.
What it does
Description of the Game Levels
Activities of Daily Living (ADLs) using the upper extremity form the core framework of our game. Each level is designed as a progressive step toward improving the arm affected by the stroke, with an ascending level of difficulty.
Heart rate elevation will be achieved by encouraging users to move their bodies through repetitive patterns. Walking plays a key role here, due to the activity of the gastrocnemius and soleus muscles, which help return blood to the heart, naturally increasing heart rate. This approach allows us to avoid using treadmills or other anaerobic equipment, helping clinicians deliver effective interventions without added costs.
Each level also includes variations in difficulty, primarily through increased repetition, to stimulate quicker and more efficient motor responses.
Level 1
The goal in Level 1 is to improve the participant’s ability to extend their arm to reach for a fruit in a basket, then grasp and transport it to the other basket on the other side of a table. This also works on trunk/core control by encouraging eccentric muscular contractions to stabilize the body during the transport movement. The task integrates reaching, grasping, and transporting skills in a functional ADL-like context.
Level 2
In Level 2, we build upon the reaching and transporting skills from Level 1 and introduce a new challenge: the participant must flex their arm to bring the fruit to their mouth. This targets both physical and cognitive functions, distinguishing between fruits (e.g., an apple vs. a banana) and performing a key self-care task: independent eating. Participants will walk around a table with a variety of fruits, locate the randomly assigned one, and bring it to their mouth as quickly as possible, repeating this multiple times to build endurance and speed.
Level 3
Level 3 combines elements from the previous levels, now introducing more dynamic challenges. Participants will pick up fruits and other kitchen-related objects placed at different heights and locations to prepare several smoothies. This requires reaching to both lower and upper levels, continuing to improve trunk control, reach, grasp, and transport, now with more functional variability and increased motor demands. IN CONSTRUCTION
Resting Periods – MI Phase
An important aspect of HIIT is including rest intervals. During these moments, we will introduce a MI phase, where the participant is prompted to mentally visualize a sequence of movements involving their affected arm while viewing a black screen. This supports movement retention and strengthens internal representations of reaching. Introducing MI between active tasks helps enhance the learning curve and prepares the user for upcoming level challenges, maintaining interest and engagement.
Heart Rate Tracking and Difficulty Adjustment
Monitoring heart rate is essential for safety and progression. The game connects to a fitness monitoring watch, tracking heart rate in real time. This data informs clinicians of the participant's current status and also signals the game when to pause or progress to the next difficulty level, ensuring a personalized and adaptive experience.
Hand Tracking
To provide objective data for clinicians, hand tracking will be integrated and recorded in every session. This enables measurement of key metrics such as maximum finger flexion, total finger angular displacement and wrist flexion. Not only is this data useful for tracking progress, but if validated, it could also serve as a built-in evaluation tool to objectively assess patient performance across sessions.
How we built it
For the Unity setup, we started with the main instructions provided in the tutorial to ensure our Oculus was pairing correctly and we didn't encounter any build errors. Next, we created a kitchen environment for the games using Unity assets to elevate the experience.
Once the environment was complete, we began working on Level 1. For this, we imported some fruits and baskets from the Unity store to set up the base of the level. We then ensured the mesh and body of the objects were compatible with hand tracking so that we could move the fruits from one basket to the other. We configured the fruits to spawn from one basket, providing the user with a basket to pick from and a goal basket to drop the fruits into. For the goal basket, we implemented a destroy function that activated once it held a certain number of fruits, preventing it from overflowing and ensuring the user could continuously add more fruits. Finally, we added a countdown and a score counter. The score counter can be displayed, but we're currently using it primarily for data collection.
For Level 2, we didn't need to import any assets as we already had the fruits. We added spawn points using empty game objects around the table to set where we wanted our fruits to appear. During this part, we ensured the cube mesh maintained a scale of (1, 1, 1) because altering the scale would flatten the fruits. We then added a destroy function that uses the Oculus camera's center of eye to determine if the fruit is close enough to the user's face when bringing it up, and if so, it would be destroyed as if the user was eating it. For the camera, we had to scale it by 2 (and also increase the size of the fruits) so that the user is forced to bring the fruit close to their face for it to be destroyed. We also added munching noises when the fruit is destroyed to create a more fun experience.
For the heart rate monitoring component, we developed an iOS app that offers two options: streaming real-time heart rate data from the Apple Watch or simulating a heart rate data sequence over time. This data is sent to a Google Firebase cloud dataset and then loaded into Unity. Once Unity retrieves the records from Firebase, the data is saved in a log file with timestamps so that physicians can review and evaluate the tests afterward.
Challenges we ran into
Vanessa Akhras: I encountered several challenges that required extensive debugging to resolve. One particular issue involved scenes not displaying correctly. For instance, I had a fruit object that wasn't scaling properly during rotation. After thorough a while, I discovered that the problem stemmed from the scaling of its parent object.
Another challenge I faced was connecting the counter to the apples moving between two different baskets. When the user moved the apples from one basket to the other, I recorded the collision between the apple and the basket. However, the collider often lost the apple tag, making it difficult to track duplicates. To address this, I needed to keep track of every apple in the goal basket, ensuring that each apple only increased the counter by one point, even if it collided multiple times with the same basket.
I also encountered issues with debugging, where I couldn't attach a process to my debugger. As a result, I had to rely on print statements to troubleshoot my problems.
Bridget: As a primarily Mac user and someone new to Unity, setting up the project and managing version control—especially when merging changes on shared scenes—were difficult. Additionally, for the heart monitoring component, the iOS app initially fetched only the first 10 historical records upon launch and then stopped updating. This issue might be due to an Apple HealthKit delay or a logic error. To work around this, I implemented a simulation mode to record synthetic data in real time.
Accomplishments that we're proud of
- A strong research framework to support our brainstorming of solutions for people with stroke.
- A solution-oriented idea addressing real-world challenges, with potential for clinical application.
- Development and preliminary testing of the first two levels.
- Testing the integration of hand tracking and heart rate monitoring with Unity.
- Clear division of responsibilities among team members to enhance productivity.
What we learned
Bridget Liu: This is my first experience with both iOS and Unity/XR development, and I'm proud that we were able to prototype a solution within the given time frame while successfully connecting the cloud database between the two applications. I gained valuable hands-on experience integrating real-time data with cloud services and learned practical lessons in problem-solving, collaboration, and effective version control—skills that will be essential in our future projects.
Vanessa Akhras: This being my first Unity project, I'm proud to have successfully created a full-scale environment with a meaningful purpose: aiding individuals who have suffered from a stroke. Throughout the project, I also learned how to troubleshoot and resolve various debugging and tracking issues. Additionally, I developed a deeper understanding of object management, collision detection, and maintaining the integrity of tags within the Unity environment.
Jonathan Akhras: I learned that it is important to optimize the skillset of each teammate. As someone who does not have coding experience, I wanted to work on my leadership skills throughout this project. I feel like I was able to put my leadership to the test and gain more experience from this hackathon.
Thiago Vidal Pereira: This was my first experience in a hackathon, and I could not be more satisfied. Besides gaining some basic perspective on the requirements to create a project using Unity, I also had contact with people from different areas that intersect with topics I am used to seeing only from the rehab perspective. This was a great experience for me to learn how to communicate knowledge in a simpler and clearer manner, enabling people to create meaningful solutions for individuals with stroke. Other learnings involved using Devpost and tailored research searching focused on product creation.
Zachary Lavoie Toure:This was also my first Hackathon. I learned many things, such as how much time debugging takes. It was my first time coding in C# and using a Unity interface. I think time management was my biggest flaw and I have really learned this during the Hackathon.
What's next for Fruitful Mind
- Finish level 3 construction.
- Improve the link between the heart rate monitoring with the game and adapt task difficulty automatically based on user performance trends.
- Add more ADL tasks (e.g., grooming, cooking, dressing).
- For clinicians or caregivers to monitor rehab progress remotely.
- Enable collaborative gameplay or supervised sessions.
- Partner with a rehab clinic or stroke support group to get feedback from actual users on usability, engagement, and comfort.
- Based on feedback, improve the game's level mechanics, and technical performance (especially hand and heart rate tracking).
- Collaborate with a research institution or hospital to run a small-scale study measuring outcomes related to usability, motivation, and basic motor function improvement.
References
Public Health Agency of Canada, using Canadian Chronic Disease Surveillance System data files contributed by provinces and territories, as of February 2021 (data up to 2017–2018). Data from Nunavut and the Northwest Territories were not available for 2017–2018.
C. D. Guerrero-Mendez, H. Rivera-Flor, A. C. Villa-Parra and T. F. Bastos-Filho, "Exploring Novel Practical Approach to Post-Stroke Upper-Limb Neurorehabilitation Based on Complex Motor Imagery Tasks," 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA, 2024, pp. 1-6, doi: 10.1109/EMBC53108.2024.10782286.
Cunha RG, Da-Silva PJ, Dos Santos Couto Paz CC, da Silva Ferreira AC, Tierra-Criollo CJ. Influence of functional task-oriented mental practice on the gait of transtibial amputees: a randomized, clinical trial. J Neuroeng Rehabil. 2017 Apr 11;14(1):28. doi: 10.1186/s12984-017-0238-x. PMID: 28399873; PMCID: PMC5387354.
Wang H, Xiong X, Zhang K, Wang X, Sun C, Zhu B, Xu Y, Fan M, Tong S, Guo X, Sun L. Motor network reorganization after motor imagery training in stroke patients with moderate to severe upper limb impairment. CNS Neurosci Ther. 2023 Feb;29(2):619-632. doi: 10.1111/cns.14065. Epub 2022 Dec 27. PMID: 36575865; PMCID: PMC9873524.
Ase, H., Honaga, K., Tani, M. et al. Effects of home-based virtual reality upper extremity rehabilitation in persons with chronic stroke: a randomized controlled trial. J NeuroEngineering Rehabil 22, 20 (2025). https://doi.org/10.1186/s12984-025-01564-5.
Laver KE, Lange B, George S, Deutsch JE, Saposnik G, Crotty M. Virtual reality for stroke rehabilitation. Cochrane Database Syst Rev. 2017 Nov 20;11(11):CD008349. doi: 10.1002/14651858.CD008349.pub4. PMID: 29156493; PMCID: PMC6485957.
Bargeri S, Scalea S, Agosta F, Banfi G, Corbetta D, Filippi M, Sarasso E, Turolla A, Castellini G, Gianola S. Effectiveness and safety of virtual reality rehabilitation after stroke: an overview of systematic reviews. EClinicalMedicine. 2023 Sep 14;64:102220. doi: 10.1016/j.eclinm.2023.102220. PMID: 37745019; PMCID: PMC10514431.
Zhang J, Jiang X, Xu Q, Cai E, Ding H. Effect of Virtual Reality-Based Training on Upper Limb Dysfunction during Post-Stroke Rehabilitation: A Meta-Analysis Combined with Meta-Regression. J Integr Neurosci. 2024 Dec 27;23(12):225. doi: 10.31083/j.jin2312225. PMID: 39735963.
Hugues N, Pellegrino C, Rivera C, Berton E, Pin-Barre C, Laurin J. Is High-Intensity Interval Training Suitable to Promote Neuroplasticity and Cognitive Functions after Stroke? Int J Mol Sci. 2021 Mar 16;22(6):3003. doi: 10.3390/ijms22063003. PMID: 33809413; PMCID: PMC7998434.
Valkenborghs SR, van Vliet P, Nilsson M, Zalewska K, Visser MM, Erickson KI, Callister R. Aerobic exercise and consecutive task-specific training (AExaCTT) for upper limb recovery after stroke: A randomized controlled pilot study. Physiother Res Int. 2019 Jul;24(3):e1775. doi: 10.1002/pri.1775. Epub 2019 Apr 3. PMID: 30942552.
Montero-Almagro G, Bernal-Utrera C, Geribaldi-Doldán N, Nunez-Abades P, Castro C, Rodriguez-Blanco C. Influence of High-Intensity Interval Training on Neuroplasticity Markers in Post-Stroke Patients: Systematic Review. J Clin Med. 2024 Mar 29;13(7):1985. doi: 10.3390/jcm13071985. PMID: 38610750; PMCID: PMC11012260.
Cucinella, S.L., de Winter, J.C.F., Grauwmeijer, E. et al. Towards personalized immersive virtual reality neurorehabilitation: a human-centered design. J NeuroEngineering Rehabil 22, 7 (2025). https://doi.org/10.1186/s12984-024-01489-5.
Log in or sign up for Devpost to join the conversation.