-
-
The home screen of the application
Inspiration
We love Computer Science(Duh) and like most other developers we never realized our greatest gift. We might not realize but our vision plays a huge role in what we do. I can't imagine debugging a code where I could not spot a missing semi-colon with my eyes. Growing up we had pseudo-languages/games that engaged us in the core concepts of programming. We had turtle, scratch etc. But what about kids with visual impairments ? How are they going to experience the same joy that we do ? Will the text editor's cold mechanic text to speech voice and hours of debugging break their morale ? Are we missing out on Genius minds just because they don't have the tools to start learning at a young age? These are the questions we asked ourselves. With all this in mind we got the inspiration needed to create this project.
What it does
Star Script is a coding platform that is targeted towards kids with visual impairment but can be used by everyone.
Start our windows application , press space bar to enter audio command release it to send it for processing, it's as simple as that.
It deploys an audio command system that allows user to create blocks in the editor. All one needs to know is which block represents which function. To make the platform truly inclusive for people we have also used speech recognition in combination with NLP(Natural Language Processing) to make the platform a hands free experience for all. Not just that, the more it is used , the more it learns and the better experience you get.
The user can navigate and code on the platform using voice commands like "start", "run", "undo", "clear", "repeat 5 times", "add", "plus", "loop 5 times", etc. With this platform you don't need to learn those boring old syntax, our program understands the 'context' of your speech, so go ahead just talk to it.
The platform also includes documentation with text-to-speech enabled, thus anyone can understand the purpose of the various coding blocks. Navigating through pages is also made for our target audience so whenever you open a new screen you will get an audio prompt.
How I built it
We designed various screens in photoshop and then implemented them in Unity. We used wit.ai for NLP and voicerrs to convert the output of programs into audio.
Challenges I ran into
Training wit.ai took a long time with the variety of options that we give it became really important to train it a lot. Further saving audio and sending it to wit.ai was trickier than expected. Also, unity doesn't support mpeg or mp3 format so we have to search a lot for an api that gave 'wav' format text to speech result.
Accomplishments that I'm proud of
We created something that is going to help a lot of young minds and we are really proud of that.
What I learned
We learned using Wit.ai with Unity and a lot of Unity fundamentals.
What's next for Star Script
The future for Star Script is to become more inclusive. Currently the platform is compatible only with the English language. We plan on making the platform multilingual thus enabling us to become inclusive of people across the globe.
Log in or sign up for Devpost to join the conversation.