TrueFit

• 

  comfyUI workflow

• 

  Chrome Extension

Inspiration

Curating personal fashion has become increasingly important as we grow older, with many seeking to express their unique styles effortlessly. Through conversations with friends and peers, we discovered that the desire for a personalized try-on experience has been around for decades—dating back to pop culture references like the movie Clueless.

Despite advancements in e-commerce, personalized try-ons remain elusive, with most current solutions relying on generic models or size charts that don’t reflect the unique fit for individual body types. This inspired us to build TrueFit, a tool that bridges this gap by bringing accurate, user-centric try-on technology directly to online shopping experiences.

What it does

TrueFit is a Chrome extension that enables users to visualize garments on themselves in real-time. Users upload 10-15 photos of themselves, and the tool integrates seamlessly with e-commerce websites like H&M or Zara. Once a garment is selected, the extension sends the data to our backend, where a Flux model personalizes the garment's appearance, ensuring an accurate fit and realistic look. Users can see how clothes fit their unique body type before buying, significantly reducing the guesswork.

How we built it

• Frontend:
  -users are to upload 10-15 personal photos to chrome extension and interact seamlessly with product pages on e-commerce websites like H&M or Zara.
  
• Manipulated DOM to showcase realtime generated images .
  

• Backend:

  • Utilized ComfyUI to orchestrate the image generation workflow, processing user-uploaded photos, selected garments, and conditioning inputs.
    
  • Flux Model Integration: Integrated the Flux 1.0 model, fine-tuned using LoRA (Low-Rank Adaptation), to adapt garments based on body proportions, poses, and individual characteristics.
    
  • Image Segmentation: Used SAM2 (Segment Anything Model) and GroundingDINO models to precisely mask garments and overlay them onto user images, ensuring garments fit naturally on the body.
    

• Upscaling and Refinement:

  • Implemented CatVTON (Virtual Try-On) techniques to refine garment alignment with the user’s pose and improve visual coherence.

• used Redis Stream to communicate between multiple workers

Challenges we ran into

Model Fine tuning: Getting the Flux model to accurately personalize garments required multiple iterations of training and fine-tuning.

• Browser Integration: Developing a seamless experience through the Chrome extension posed challenges, we had to deal with connecting our multiple cloud services to one chrome extension.
• The comfyUI framework was quite difficult to integrate with other extensions. We used this framework primary because it was highly suggested by the CV community.

Accomplishments that we're proud of

• This was our first end to end project working with image generation models!
• It was exciting building something that actually can be used by our peers for a valid issue!

What we learned

• This was our first experience working with ComfyUI. We learned to navigate its node-based system, integrate external models like Flux and LoRA, and build complex image pipelines efficiently!

What's next for TrueFit

• We would love to take this from an MVP to an actual product. We'll probably start doing more user interviews to see what exact functionalities people want and build in that direction.
• It would be great to expand try on features to accessories + clothes to get a more dynamic experience

Built With

• comfyui
• javascript
• python
• redis