BlockBuddies

BlockBuddy running around autonomously.
BlockBuddy planting a plant.

Inspiration

The idea for Block Buddies came from the desire to combine two things we love: gaming and AI. We saw how AI is transforming different industries and realized it had the potential to create meaningful, personal experiences in gaming. The rise of AI companions in various apps and entertainment sparked our curiosity—what if gamers could have a truly adaptive, interactive friend to team up with whenever they wanted? This led us down the path of building Block Buddies.

What it does

Block Buddies is an AI companion built specifically for Minecraft players. It’s your in-game friend that helps with building, exploring, and surviving, offering real-time tips, conversation, and support as you play. Whether you need help gathering resources, fending off mobs, or just want a chat while mining, Block Buddies is there to make sure you never adventure alone in Minecraft.

How we built it

Our tech stack consists of AWS Bedrock, AWS EC2, LlamaIndex and Next JS. Using these, we have created fully automated agent-driven players.

Challenges we ran into

Our number one problem was, how we can represent the 3d world for an LLM whose input/output is one dimensional. To solve this, we used vector embeddings, and turned the player agent's surroundings into a vector representation.

Accomplishments that we're proud of

We solved the 3D environmental embedding problem with minimal performance impact. Our agents are able to understand, traverse and interact with their environment. We also created a 3D visualization of the agent's environment and our custom embeddings.

What we learned

This was our first time working with LlamaIndex and AWS Bedrock. We managed to setup our agents with custom tools to interact with the Minecraft world.