Meta4All

title
the problem
our solution
demo video
future extensions
thanks!

Inspiration

Many companies such as Meta (formerly Facebook) are attempting to integrate virtual reality into our daily lives through the “Metaverse”. News outlets like the Wall Street Journal have acknowledged the Metaverse as the future of the internet. Unfortunately, those that are visually impaired would be unable to participate in these virtual spaces, further excluding them from our digital society. Our mission is to make VR, and by extension, the future Metaverse more accessible for those that are visually impaired

What it does

Our project is a screen reader, made for 3D environments like virtual reality worlds. It analyzes the user’s digital surroundings and synthesizes this data into natural language which is expressed in audio through text-to-speech software. It takes into account the user’s relative orientation in the world and therefore allows visually impaired people to navigate and participate in social VR experiences.

How we built it

We built our proof of concept on Roblox Studio, as it is a digital social space similar to current virtual reality technology. This required us to use the scripting language Lua. In order to convert text to speech, we used Google Cloud’s Text To Speech Service.

Accomplishments that we're proud of

Our team members have little experience with Roblox Studio and Lua, and it was a rewarding experience to be able to prototype our idea within a few hours. It took some complex linear algebra calculations to turn the 3D numerical data to natural language.

What's next for Meta4All

We are excited to try implementing our technology in other, more complex, virtual reality worlds such as VRChat. We think that incorporating AI into Meta4All will allow it to describe more intricate environments, something that will be necessary as VR continues to develop. In addition, we would like to use spatial audio to further help orient the user and further immerse them into the world.