AiRNote

Inspiration

We picked one of the cutting-edge technology, Augmented Reality, to be the interaction that some users might find less accessible. AR provides an immersive social experience where users can visualize things that are not possible in mere reality. We also wanted to create a new way of communication and collaboration, such that it is simple to use, easy to engage and closely connects people. The most familiar and down-to-earth way of collaboration is through a whiteboard full of sticky notes. Hence, we have finalized our application to be the AiRNote.

What it does

AiRNote allows users to generate, edit, and customize AR sticky notes and attach them to a surface. The surface could be on a wall, on the floor, on the table, or simply on their laptops. Users can generate the notes by simply speaking, and when finished, simply click "Add Note" to generate a new note. The coolest thing of AiRNote is that it allows several different users/devices to share the AR field such that multiple people can view the same set of sticky notes/photos. Thus, people can easily collaborate and communicate without worrying about missing a single note of thoughts.

How we built it

We built the app natively on iOS using Swift, with the use of ARKit, RealityKit, SwiftUI, and the modeling software Blender.

Challenges we ran into

There are lots of things we tried for the first time, hence a bit challenging for those parts: first time to write a native iOS app using ARKit and RealityKit, first time using Blender to model something used in production, first time collaborating on an Xcode project remotely. The API of ARKit and RealityKit, as well as using the Blender software, required some learning curve; the adaption of both SwiftUI and UIKit became an issue; the Xcode project file could corrupt and be hard to recover when there is a merge conflict.

Accomplishments that we're proud of

It's an AR app!
It's the first time we modeled something and used it in production! (RealityKit)
It's using Multipeer sharing, which enables an extremely immersive and collaborative experience!

What we learned

Swift, SwiftUI, ARKit, RealityKit, etc.
How to collaborate on an Xcode project
How to do modeling using Blender
Apple Multipeer Connectivity API

What's next for AiRNote

For the next stage of development, we aim to add more ways/functionalities of interacting with other users: e.g. ability to add drawings, share files, or even the ability to share any AR object in the session.