RekindleAR

Bringing Memories Back to Life

RekindleAR Image

📖 Overview

RekindleAR is a pioneering augmented reality (AR) tool for enabling people with Alzheimer’s, dementia, and other memory-loss-related conditions to reconnect with memories they may have forgotten. By integrating advanced technologies, RekindleAR redefines the way people experience memories in an immersive environment, utilizing Spectacles by Snapchat. Our mission is to equip people suffering from memory-related issues with an AR experience that helps them engage with their loved ones and lost memories, ultimately improving their happiness, memory recollection, and reducing feelings of isolation.

🚀 Inspiration

The journey to create RekindleAR was fueled by a profound understanding of current memory-loss-related disorders. Alzheimer’s disease, specifically, is a leading cause of dementia that affects millions of people worldwide, gradually robbing individuals of their memory, cognitive functions, and sense of identity. As the disease progresses, patients often lose the ability to recognize their surroundings, loved ones, and significant personal objects, leading to confusion, anxiety, and isolation. This impacts the quality of life for patients and places an emotional burden on caregivers and family members. While current therapeutic options focus on pharmacological treatments and traditional cognitive therapy, there is a growing need for innovative, non-invasive solutions to assist with memory recall and cognitive rehabilitation.

🎯 What It Does

RekindleAR offers a comprehensive suite of features designed to enhance the experience of virtually reliving memories and seeing loved ones:

Real-time AR Overlays: Allows users to see their loved ones virtually in front of them, displaying videos and audios based on the person the user is viewing.
Machine Learning Integration: A web app that integrates with cloud storage and APIs to run facial recognition and pattern-tracking algorithms for this multi-modal experience.
Caregiver Application: An iOS application for family and caregivers to add their photos and audio for their elderly loved one’s AR experience, along with memory tracking for recognition of loved ones and health data.
Safety Features: Spectacles offer a lightweight and seamless AR experience and can be customized to the user’s vision and physical abilities.

🛠 How We Built It

RekindleAR is the result of collaborative innovation, employing a variety of technologies:

Languages:
- Python for backend and image classification processing
- Terraform for cloud deployment
- Swift for iOS app development
- HTML/CSS/JS for web app and UI/UX
- TypeScript for AR development
Frameworks:
- Lens Studio: Snapchat Spectacles AR development
- PyTorch: Machine learning model development
- OpenCV: Image processing for face detection
- Terraform: Backend API and cloud server for real-time data handling
- Xcode: iOS App Development
Platforms:
- iOS for mobile accessibility
- Web app for model accessibility
Cloud Services:
- Google Cloud and Terraform for scalable model training and data processing
APIs:
- Custom API for secure deployment and integration of machine learning models
Hardware:
- Snapchat Spectacles for easy and portable AR viewing for the elderly

🧗‍♂️ Challenges We Faced

The path to creating RekindleAR was not without its hurdles:

Learning Lens Studio: Achieving seamless synchronization between AR overlays of images and memories and real-time user motion was challenging. We focused on utilizing Lens Studio to the best of our ability and learning new features of AR development.
Model Accuracy: Training our machine learning models required extensive data and computational resources. Ensuring high precision while maintaining speed was critical.
User Interface Design: Crafting an AR interface that is both informative and easy to interact with proved to be a significant design challenge. We aimed to balance functionality with user experience.

🎉 Accomplishments We're Proud Of

We take pride in several milestones achieved throughout the development of RekindleAR:

Successful Integration: Merging AR technology with real-time user motion and integrating iOS and web apps created a tool that gives the experience of reliving and remembering memories in real-time along with analytics and data behind the scenes.
Advanced Machine Learning Models: We developed a highly accurate facial detection model that operates efficiently in real-time environments and over the cloud, along with a dataset of diverse facial features.
User-Friendly Design: Our simple yet effective interface design provides real-time memory viewing, allowing users of any age and physical ability to participate.

📚 What We Learned

The creation of RekindleAR offered profound insights into the intersection of technology and user experience:

Integration Techniques: We learned effective strategies for incorporating machine learning models into AR applications.
Data Management: Handling large facial datasets taught us the importance of data integrity and processing speed.
User-Centered Design: We gained valuable experience in designing interfaces for user-friendly environments, ensuring usability and accessibility.

🔮 What's Next for RekindleAR

The journey doesn't end here! Our future plans for RekindleAR include:

Data Collection: Testing our system in real user settings to gather valuable feedback and refine our technology.
ML Model Expansion: Enhancing our machine learning capabilities to detect a broader spectrum of faces and groups of people.
AR Feature Development: Incorporating advanced geolocation features and in-tool AR scanning to retrieve memories and audios based on locations and images scanned.
Hardware Integration: Adding functionality for gyroscope tracking to sense a spike in motion, which could detect potential seizures.

🛠 Built With

Languages: Python, Swift, HTML, CSS, JS
Frameworks: PyTorch, OpenCV, SwiftUI, Lens Studio
Platforms: iOS, Web app
Cloud Services: Terraform
APIs: Custom API