AI1y | Devpost

The screen reader chrome extension

Inspiration

We wanted to build a screen reader using AI that helps visually impaired users navigate web pages more fluidly. Existing screen readers often lack contextual understanding and can feel disjointed. We were driven to create a tool that offers a more human-like and seamless experience.

What it does

AI1y is a Chrome extension that enhances web accessibility by:

Generating image descriptions using AI.
Summarizing entire pages into digestible insights.
Allowing users to ask questions and automatically scroll to the relevant sections.
Providing a structural overview of the webpage, helping users understand layout and content flow.

How we built it

We built AI1y as a Chrome extension using vanilla JavaScript. We integrated AI APIs to analyze visual and textual content in real-time and used the Web Speech API for text-to-speech functionality.

Challenges we ran into

Designing a flexible and robust text-to-speech (TTS) logic that adapts to different types of content.
Working around browser security policies that restrict certain DOM manipulations or API calls.
Ensuring smooth navigation and interaction without overwhelming the user.

Accomplishments that we're proud of

Successfully generating meaningful image descriptions using AI.
Enabling accurate and concise summarization of complex pages.
Allowing users to engage with the page through natural language questions and get contextual answers with automatic navigation.

What we learned

How to architect a screen reader from scratch.
The nuances of building Chrome extensions.
Practical experience with integrating AI tools for real-world accessibility use cases.

What's next for AI1y

We aim to make AI1y fully voice-controlled, eliminating the need for keyboard or mouse interaction. We're also exploring voice-based password authentication to enhance security and personalization.