LandMarkGo!

LandMarkGo!

Inspiration

We were inspired by other apps like pokemon go where users can interact with landmarks like pokestops and save them to their library. We designed the map feature to make this cultural experience viable walking through the city, creating a quick and fun real world immersion.

What it does

The project's features include landmark identification through live photos and info such as a description, fun facts, and historical context. The map feature automatically tracks your location and allows you to find landmarks close, enriching local culture. Every picture taken is saved in a library, allowing users to reflect on their journeys.

How we built it

We used javascript, react native, and HTML/CSS for the frontend. gemini API and google vision API powered our accurate and detailed image and description analysis. Google maps API allowed us to find quick routes to local landmarks.

Challenges we ran into

We had trouble parsing gemini's response to our homepage, making it hard for users to interpret the landmark description. We resolved this issue. A second issue was having google vision recognize landmarks. Since this feature is best used with a live feed of the real world, we had to match 2D planes, change resolution, and emply other techniques to have google vision interpret the landmarks.

Accomplishments that we're proud of

We are especially proud of the gemini, vision, and map features. These were hard to implement but rewarding, as we are able to quickly and consistently identify landmarks and route to them, making a seamless user experience.

What we learned

We learned how to employ google vision for the first time, from the process of registering and learning tools on cloud console to fetching its image analysis results. We combined these results with gemini to add more context and detail, providing a unique approach. We had fun learning react features like text type as well.

What's next for LandMarkGo!

We plan to include IOS integration to allow even more portable analysis where users can easily take snaps on the app that crossover to their iphone's photos app. We want to create a more robust network and backend where users save data storage on a database with other users they can friend request and share libraries with.