Overview
In this project, out team tried generating multiple styles from a single image. The goal was to apply current deep learning technologie for creating stylized portraits from camera captured picture.
My Work
The main goal of this project was divided into two parts. The first task if to do face-swapping from captured pictures to some templates with certain poses. The second task was to stylize the swapped result to create multiple stylized portraits.
For the first task, I used the paper of E4S to implement face swapping. The E4S's framework proposed a new approach to combine GAN with a mask-guided injection module to synthesize the swapped face. This satisfies our team's need to fine-tune some facial features during the face-changing process.
For the second task, I used the paper of OSASIS to implement stylization. Their approach aims to achieve effective stylization by initially disentangling the structural and semantic information of images. Especially, by combining the advantages of GAN and Diffusion Models, OSASIS is robust in structure preservation, allowing it to control the level of content and style implemented to a given input. This satisfies out team's need to preserve fine-tuned facial features and makes us easy to control the stylization process.
Outcome
In the end, although we were unable to complete the entire project due to time constraints, I still learns a lot from this project.
By referring to and applying the E4S and OSASIS models, I gained a deeper understanding of the fundamental principles of image generation and style transfer, as well as these deep-learning approaches like GAN and Diffusion Models and the usage of pyTorch.
The E4S face-swapping model helped me understand how to achieve realistic face replacement while preserving the characteristics of the original image, while the OSASIS model helped me understand stylizations and the implementation of multi-style transfer.
Through this project, I gained a deeper understanding of the fundamental principles of image generation and style transfer, and furthered my knowledge of related deep learning techniques and methods.
