AR Bae

Personalised Face
Body Generation using smplx
Face mask generation using NextFace
Blender animation

Inspiration

Whenever we try on clothes, it's always on mannequins that don't look like us. We're not sure if the clothes would look as good on us as they do on the mannequins. We thought, what if there was a way to see what the clothes would look like on you without having to try them on? What if you could see a realistic 3D image of yourself wearing the clothes from any angle? This would be a great way to shop for clothes online or in-store. Also, this would be a very challenging task to accomplish in the limited time available, but all of us were super pumped to take this challenge head-on.

What it does

We have tried to make this virtual try-on feature, which can make 3D models of the user using AI. It will do a physical simulation of the clothes on the users and will let them feel every wrinkle of that cloth on their body in the 3D Virtual Avatar. You would be able to see what the clothes look like on you before you even buy them. This will help you in improving not only your purchase decisions but also validate them. You'll be able to easily navigate through the big wardrobes of e-commerce clothes and find the things that will look good on you just by clicking a picture of yourself.

How we built it and the challenges we ran into

To build this project, we started with Unity, a 3D engine that can be used for making games, but it is also useful for simulating things and building an interface to connect everything. In other words, it helps us to connect the dots. We initially had a lot of trouble creating our characters, and even after we managed to create them, the cloth simulation wasn't as advanced as we had hoped and required a lot of manual intervention. So we decided to try using Unreal Engine. However, its complexity can make it challenging to learn, especially in a short span of 48 hours. After investigating other software options, the team decided that Blender would better fit our needs. We generated a model in Unity and imported it into Blender to simulate cloth physics. This entire pipeline was built using Python, Unity, and Blender. The entire process of body generation takes about 20 minutes, while the rendering of individual clothes on the body takes around 2 minutes. Following this, the user gets an entire animation of how a piece of cloth will look on them, with the perfection of every wrinkle of the cloth.