Ah Boy

• 

  Demo Screenshot

• GIF

  Demo Video

Inspiration

In a world of rapid digitalisation and ageing, friction occurs as less receptive seniors lack the skills necessary to interact with digital tools. From our personal experiences and extensive research, the elderly, especially those whom speak dialect exclusively, are increasingly disconnected from these tools simply due to their complexity. One example is ChatGPT, where inputs and outputs have to be typed and English by default. That got us thinking - can there exist a service that can help bridge the existing gap, so that a substantial group of the elderly can benefit from the same technologies many of us use on a daily basis?

What it does

Ah Boy is an innovative speech-to-text chatbot designed with ease of use in mind. By supporting spoken languages like Hokkien, dialect-speaking elderlies now have an integrated application they can easily use to take advantage of to answer all of their queries.

How we built it

Our front-end interface is largely built using frameworks and tools like React and Chakra UI. On the other hand, our pipeline consists of open-source speech-to-text and text-to-text APIs provided by developers like Meta, with the GPT AI being our chat engine of choice. Response from these APIs are subsequently passed to our front-end using the same tools.

Challenges we ran into

The technologies and interfaces we required to build our chatbot were fragmented and thus, having to integrate our pipeline from the user speech input to the OpenAI GPT text output required us to meticulously layer our tasks and perform debugging tests to figure out the optimal method of delivering our product.

Accomplishments that we're proud of

Our app is the first Hokkien centric chatbot implemented with elderly-friendly features like text size adjustment, thus filling the critical void left in the wake of our digitalising world.

What we learned

Throughout the entire project frame, we learnt massively about the management of our project in terms of work delegation, time management and more. This helped us step outside our comfort zone of handling technical matters to look at the bigger picture of our product.

Delving into the technical aspects of our project, we learnt a great deal on implementing ReactJS and ViteJS to create a seamless front and back-end integration. It was also our first time utilizing the OpenAI GPT API as well as Meta's new Hokkien translator API.

What's next for Ah Boy

We are looking towards implementing the technologies for a speech output in addition to the current text output, as well as providing support for more dialects and/or languages like Cantonese. Features to enhance the user experience such as chat history, will also be implemented.

Built With

• javascript
• openai
• python
• react
• vitejs