ElderSpeak

Website
IOT Speaker Device

Inspiration

We were inspired by certain home assistants, as we saw potential in them being chatbots/good company, but are currently not being utilised as so, many use it simply for the concept of a smart home. The problem statement we came up with was also one concerning to us, most of which are worried about the elderly in our society, especially our family members being lonely since most of them live alone.

What the product does

Our product can communicate act as a companion for all users, mostly targeted at senior citizens. It has many features, such as reminders for medicine/important dates, we also have an option to summarise the whole conversation the person had with the AI. With these features, we hope to elevate users mental well-being, organise their daily life and just alleviate their loneliness.

How we built it

Using a multitude of APIs, such as google's vertex AI, and their text to speech and Next.js, we managed to build this product

Challenges we ran into

The API did not have a built in feature that allowed it to store the past conversations, which we managed to fix by feeding it its past replies and messages. The voice to speech was also slightly iffy, but we managed to refine it to become slightly more accurate.

Furthermore, the IOT implementation with the ES32 was too complicated and we could not finished in time, so we opted for a API backend to linkup with future IOT devices

Accomplishments:

We managed to integrate cutting edge language model and text-to-speech APIs, improve/add on it and finally create a working prototype for ElderSpeak. we also managed to integrate state-of-the-art hardware resources to make ElderSpeak a real product for consumers to use

What we learned

We learnt how to access and properly use AI APIs, most notably Google's Vertex AI and of course OpenAI's GPT API and integrate them into our projects. we also learnt more about deep learning and its endless capabilities and how we can further improve on language models simply by prompting it with natural languages.

Whats Next:

We plan for a step by step process, aimed at greatly improving UI/UX of ElderSpeak:

Implementation of an ESP32 or other IOT devices and physical speaker model instead of a webpage for better audio inputs
Usage of virtual avatar for ElderSpeak
Verbal start/stop keywords instead of physical buttons
Inclusion of touchpad for more inclusivity by utilising handwriting inputs.
Realistic voice outputs using AI voice models
Potential use of cloud services for data storage to establish a SaaS