Smart Cataloging

After processing using Gemini API on the front-end.
Fine tuned YOLO V11 nano performance on dataset.

Inspiration

Small retailers face significant challenges when trying to list hundreds of products on e-commerce platforms. Limited time, technical expertise, and the sheer volume of SKUs make the process daunting. We were inspired to simplify this process by leveraging technology to automate catalog creation using shelf images.

What it does

Our solution uses shelf images to identify products, segment them, and generate a structured product catalog. Retailers can review, edit, and finalize their catalog through a user-friendly interface, making onboarding fast and efficient.

How we built it

We used YOLOv11 Nano for real-time image segmentation, breaking shelf images into smaller product-level snapshots. These snapshots are then processed by the Gemini model, which combines OCR, fuzzy search, and context-aware image recognition to identify products. Finally, we present the data as a JSON catalog, which retailers can interact with on our web platform.

Challenges we ran into

We faced issues with SKU variability, missing or unclear labels, and products with similar packaging. Balancing speed, accuracy, and cost-effectiveness was another challenge, especially given the diverse needs of small retailers.

Accomplishments that we’re proud of

We successfully created a near-instant product cataloging pipeline that combines segmentation, recognition, and user interaction. The solution is both efficient and accessible, even for non-tech-savvy retailers.

What we learned

We learned how to optimize AI models for real-world retail scenarios, tackle edge cases like ambiguous product labels, and design a user-friendly experience that adds value to small businesses.

What’s next for Smart Cataloging

We aim to enhance accuracy by integrating barcode recognition and expanding our dataset to support regional products. We also plan to introduce multi-language OCR and scale the solution to handle diverse store types, making it even more inclusive.