PerfectBlend

Project Inspiration

At my first co-op, I am helping fine-tune an instance segmentation model to count marine species from drop camera surveys around the Belcher Islands to support local fisheries and conservation efforts. One problem we ran into was class imbalance. Our annotated dataset had far more scallops and clams than fish species. If left uncorrected, the model becomes confident on majority classes and blind to rare ones. Fixing this manually meant writing one-off scripts, guessing at target distributions, and hoping the augmentation didn't corrupt the annotation geometry.

What if there was a tooI that made this process guided, visual, and intelligent?

Product Summary

PerfectBlend is an MLOps tool that diagnoses and fixes class bias in computer vision datasets, guided by generative AI.

A user connects a local COCO-format CV dataset by path. PerfectBlend parses every annotation at the per-image level and renders a packed bubble chart, creating a data visualization that intuitively displays how skewed the dataset really is.

The user then describes the purpose of their model (e.g., "detecting marine species in underwater surveys for conservation" or “identifying and diagnosing skin conditions for patients”). This context is sent to a Railtracks agentic pipeline connected to Gemini, which reasons about what the right class balance should look like for that specific use case, grounded in real-world domain citations.

The AI returns per-class strategies with target annotation counts. The user can review and manually adjust every target before applying, a human-in-the-loop checkpoint that keeps the user in control.

PerfectBlend then physically rebalances the dataset, either by downsampling majority classes by removing images or upsampling minority classes through augmentation (geometric transforms, recolouring, noise) while keeping COCO annotation geometry (both bounding boxes and segmentation masks are supported) correctly transformed. The output is a new, balanced dataset.

Results are shown side-by-side as interactive before/after bubble charts with detailed statistics regarding changes in class size.

Technology Stack

Frontend: React, TypeScript, Tailwind CSS, D3.js (data visualizations) Backend: Python, FastAPI, Railtracks, Gemini API, Albumentations (image augmentation library)

AI Use

Yes

Accomplishments that we're proud of

Training a RF-DETR detection model on both the original and rebalanced versions of a skin lesion dataset and observing mAP improve from 0.56 to 0.65 with no changes to the model or training procedure. Training a marine organism instance segmentation model saw mAP improve from 0.38 to 0.44. These quantitative results validate that this tool does what it claims.

Additionally, the human-in-the-loop design was important to get right. The AI suggests targets, and the researcher sees exactly what will change and can override every number before a single file is touched.

What's next?

• Support for additional annotation formats such as YOLO and Pascal VOC
• The ability to configure the augmentation strategy, allowing user to select which transformations to apply

Built With

• albumentations
• d3.js
• fastapi
• gemini
• python
• railtrack
• react
• typescript