Image-based virtual try-on involves synthesizing a tryon
output where the target model is wearing the in-shop garment
while other characteristics of the model and garment are preserved.
The above output is generated by our proposed method ZFlow.
Abstract
Image-based virtual try-on ok involves synthesising perceptually convincing images of a model wearing a particular garment and has garnered significant research interest due to its immense practical applicability. Recent methods involve a two stage process: i) warping of the garment to align with the model ii) texture fusion of the warped garment and target model to generate the try-on output. Issues arise due to the non-rigid nature of garments and the lack of geometric information about the model or the garment. It often results in improper rendering of granular details. We propose ZFlow, an end-to-end framework, which seeks to alleviate these concerns regarding geometric and textural integrity (such as pose, depth-ordering, skin and neckline reproduction) through a combination of gated aggregation of hierarchical flow estimates termed Gated Appearance Flow, and dense structural priors at various stage of the network. ZFlow achieves state-of-the-art results as observed qualitatively, and on quantitative benchmarks of image quality (PSNR, SSIM, and FID). The paper presents extensive comparisons with other existing solutions including a detailed user study and ablation studies to gauge the effect of each of our contributions on multiple datasets
Network architecture
ZFlow comprises of two modules: A) Garment Warping to deform the garment Ip to align with model Im and generates warped
garment (Iwrp), and B) Texture Fusion which has two sub-steps - i. Conditional Segmentation to predict a post try-on clothing segmentation
of the model Mexp ii. Dense Fusion which combines the warped garment (Iwrp) and segmentation mask (Mexp) to generate the final output
(Itryon). Gated Appearance Flow for garment warping improves textural integrity of Itryon by regularizing the per-pixel flow estimation.
Dense geometric priors Ipriors improves geometric integrity of the try-on output.
Resources
- Paper: ICCV
- Supplementary: ICCV_supplementary
- Video: YouTube
- Results for comparison: Zip file
Citation
@InProceedings{Chopra_2021_ICCV,
author = {Chopra, Ayush and Jain, Rishabh and Hemani, Mayur and Krishnamurthy, Balaji},
title = {ZFlow: Gated Appearance Flow-Based Virtual Try-On With 3D Priors},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2021},
pages = {5433-5442}
}