BattleTipsAssistant

Multiplayer AR Battle Environment with AI assistant that when called upon gives the user a suggestion on a "move" or move type to choose depending on the situation.

Comment

Inspiration

Traditional tutorials in games often fall short, proving ineffective and difficult to implement especially in AR application. We were inspired to create a solution that offers players real-time assistance without disrupting the flow of gameplay.

What it does

BattleTipsAssistant allows two players to engage in a multiplayer augmented reality (AR) battle arena. If a player needs help making a decision during the battle, they can tap the "Help" button. This action sends the current battle information to an AI model, which queries a LlamaIndex to provide useful battle tips and guidance.

How we built it

We developed the AR experience using Unity. The AI model is hosted on a HuggingFace Space and is invoked through a C# script within the Unity project whenever assistance is requested.

Challenges we ran into

One of the main challenges was anchoring the world space consistently for both players in the AR environment. Additionally, with very limited experience in AI, deploying a model and utilizing a LlamaIndex presented a steep learning curve but was ultimately manageable.

Accomplishments that we're proud of

We're proud to have successfully integrated a Retrieval-Augmented Generation (RAG) pipeline into a multiplayer AR application. Despite time constraints, we worked efficiently to build a functioning project that combines these complex technologies, which often don't integrate seamlessly.

What we learned

We expanded our knowledge significantly in the realm of AI, learning what a RAG model is and how to deploy one effectively. Setting up a simple RAG workflow was a valuable experience. We also enhanced our skills in using Unity for creating multiplayer AR experiences.

What's next for BattleTipsAssistant

Moving forward, we plan to enrich the game by adding more mechanics. These additions will be incorporated into the LlamaIndex, enabling the assistant to offer even more relevant and helpful tips during battles, thereby improving the overall player experience.

Built With

  • cohere
  • fastapi
  • huggingface
  • llamaindex
  • photon
  • unity
Share this project:

Updates