Handar

Inspiration

The inspiration for HANDAR came from observing the challenges people face when providing remote assistance, especially in scenarios requiring precise instructions and guidance. We envisioned a tool that could make remote help as effective and intuitive as being physically present.

What it does

HANDAR is an interactive AR video call app that detects and visualizes 3D hands and objects in real-time. It allows users to provide remote assistance with added annotations, making instructions clear and actionable.

How we built it

We built HANDAR using Unity and integrated WebRTC for streaming capabilities. We utilized MediaPipe for hand and object detection, and Unity's AR Toolkit for overlaying 3D models in the video feed. Our team collaborated on designing an intuitive user interface and seamless interaction flow.

Challenges we ran into

One of the main challenges was achieving real-time detection and rendering of 3D hands without significant latency. Integrating WebRTC with Unity and ensuring stable streaming quality was another hurdle. Additionally, creating an intuitive annotation system that works smoothly in an AR environment required careful design and testing.

Accomplishments that we're proud of

We are proud of successfully implementing real-time 3D hand detection and AR annotations, which significantly enhance the remote assistance experience. Our seamless integration of WebRTC for video streaming in Unity was a major technical achievement.

What we learned

Throughout the development of HANDAR, we gained deeper insights into AR technology and real-time video processing. We learned about the complexities of integrating different technologies and optimizing performance for a smooth user experience. Collaborative problem-solving and iterative testing were crucial to our success.

What's next for Handar

Next, we aim to refine HANDAR's user interface and expand its capabilities by adding more interactive tools and features. We plan to conduct user testing to gather feedback and improve the app's usability. Exploring potential integrations with other platforms and expanding the app's reach to different industries are also on our roadmap.