Inspiration
We were inspired by online communities such as r/place, which fostered a vibrant and collaborative fanbase with nothing more than pixel art. In particular, the idea of social connectivity being fostered through the medium of art, and we wanted to recreate this sense of connection at Simon Fraser University, but in a more technologically advanced way.
What it does
SFU-Place is an iOS app that uses augmented reality with ARKit to creating art on virtual canvases located at various spots on campus, which can then be seen by other students using the app. Canvases can be viewed by scanning QR codes located at different locations on campus.
How we built it
The main drawing portion of the app is made with Unity, which handles drawing and plane detection - complete with 6 colors to choose from and an eraser mode. The iOS app is built with Swift and interacts with a remote server made with Flask and Firebase, handling user authentication and all database interactions whilst embedding the Unity app. The Swift app handles all API calls and embedding the Unity app, passing data from the server to Unity.
Challenges we ran into
Because there were three main components of the entire app (Swift wrapper, Unity app, and backend), it was challenging to figure out how they communicated and how it would all be put together. Another challenge we ran into was trying to persist brushstrokes on surfaces between sessions - which involved a Unity->Swift->Firebase pipeline and vice versa. We also couldn't figure out how to map each room in a permanent way to help with persistence.
Accomplishments that we're proud of
Deploying the remote server such that the iOS app could interact with it, implementing the ability to draw on the canvases and have it remain in-place, connecting the Unity app, iOS app, and server together such that data could be loaded into the Swift app from the server.
What we learned
We learned how to develop apps for iOS with Swift, how to use ARKit with Unity to get real-time direct interaction with augmented reality, and how to embed Unity into a Swift app
What's next for SFU-Place
Possibly adding new features such as different types of ways to draw, integrating our eraser functionality that we drafted, user tags for drawings, and modifying the current system to get actual real-time data by establishing a connection between iOS app and server. An Android port using ARCore is something we are considering as well.
Log in or sign up for Devpost to join the conversation.