Synthesization and reconstruction of 3d facesby deep neural networks
Başlık çevirisi mevcut değil.
- Tez No: 720406
- Danışmanlar: DR. STEFANOS ZAFEİRİOU
- Tez Türü: Doktora
- Konular: Biyoteknoloji, Biotechnology
- Anahtar Kelimeler: Belirtilmemiş.
- Yıl: 2020
- Dil: İngilizce
- Üniversite: University of London
- Enstitü: Yurtdışı Enstitü
- Ana Bilim Dalı: Belirtilmemiş.
- Bilim Dalı: Belirtilmemiş.
- Sayfa Sayısı: 167
Özet
Özet yok.
Özet (Çeviri)
The past few decades have witnessed substantial progress towards 3D facial modelling and reconstruction as it is high importance for many computer vision and graphics applications including Augmented/Virtual Reality (AR/VR), computer games, movie post-production, image/video editing, medical applications, etc. In the traditional approaches, facial texture and shape are represented as triangle mesh that can cover identity and expression variation with non-rigid deformation. A dataset of 3D face scans is then densely registered into a common topology in order to construct a linear statistical model. Such models are called 3D Morphable Models (3DMMs) and can be used for 3D face synthesization or reconstruction by a single or few 2D face images. The works presented in this thesis focus on the modernization of these traditional techniques in the light of recent advances of deep learning and thanks to the availability of large-scale datasets. Ever since the introduction of 3DMMs by over two decades, there has been a lot of progress on it and they are still considered as one of the best methodologies to model 3D faces. Nevertheless, there are still several aspects of it that need to be upgraded to the“deep era”. Firstly, the conventional 3DMMs are built by linear statistical approaches such as Principal Component Analysis (PCA) which omits high-frequency information by its nature. While this does not curtail shape, which is often smooth in the original data, texture models are heavily afflicted by losing high-frequency details and photorealism. Secondly, the existing 3DMM fitting approaches rely on very primitive (i.e. RGB values, sparse landmarks) or hand-crafted features (i.e. HOG, SIFT) as supervision that are sensitive to“in-the-wild”images (i.e. lighting, pose, occlusion), or somewhat missing identity/expression resemblance with the target image. Finally, shape, texture, and expression modalities are separately modelled by ignoring the correlation among them, placing a fundamental limit to the synthesization of semantically meaningful 3D faces. Moreover, photorealistic 3D face synthesis has not been studied thoroughly in the literature. This thesis attempts to address the above-mentioned issues by harnessing the power of deep neural network and generative adversarial networks as explained below: Due to the linear texture models, many of the state-of-the-art methods are still not capable of reconstructing facial textures with high-frequency details. For this, we take a radically different approach and build a high-quality texture model by Generative Adversarial Networks (GANs) that preserves details. That is, we utilize GANs to train a very powerful generator of facial texture in the UV space. And then show that it is possible to employ this generator network as a statistical texture prior to 3DMM fitting. The resulting texture reconstructions are plausible and photorealistic as GANs are faithful to the real-data distribution in both low- and high- frequency domains. Then, we revisit the conventional 3DMM fitting approaches making use of non-linear optimization to find the optimal latent parameters that best reconstruct the test image but under a new perspective. We propose to optimize the parameters with the supervision of pretrained deep identity features through our end-to-end differentiable framework. In order to be robust towards initialization and expedite the fitting process, we also propose a novel self-supervised regression-based approach. We demonstrate excellent 3D face reconstructions that are photorealistic and identity preserving and achieve for the first time, to the best of our knowledge, facial texture reconstruction with high-frequency details. In order to extend the non-linear texture model for photo-realistic 3D face synthesis, we present a methodology that generates high-quality texture, shape, and normals jointly. To do so, we propose a novel GAN that can generate data from different modalities while exploiting their correlations. Furthermore, we demonstrate how we can condition the generation on the expression and create faces with various facial expressions. Additionally, we study another approach for photo-realistic face synthesis by 3D guidance. This study proposes to generate 3D faces by linear 3DMM and then augment their 2D rendering by an image-to-image translation network to the photorealistic face domain. Both works demonstrate excellent photorealistic face synthesis and show that the generated faces are improving face recognition benchmarks as synthetic training data. Finally, we study expression reconstruction for personalized 3D face models where we improve generalization and robustness of expression encoding. First, we propose a 3D augmentation approach on 2D head-mounted camera images to increase robustness to perspective changes. And, we also propose to train generic expression encoder network by populating the number of identities with a novel multi-id personalized model training architecture in a self-supervised manner. Both approaches show promising results in both qualitative and quantitative experiments
Benzer Tezler
- Taşıyıcısız immobilizasyon yöntemiyle çapraz bağlı glukoz oksidaz (GOD) enzim agregatlarının ve kristallerinin sentezi ve karakterizasyonu
Synthesization and characterization of cross-linking glucose oxidase (GOD) enzyme aggregates and crystals with carrier-free immobilization methods
YASEMİN İSPİRLİ
- WC esaslı sert malzemelerin spark plazma sinterleme yöntemiyle geliştirilmesi
Improving of WC based hard materials by spark plasma sintering
EMRAH ARSLANKARA
Yüksek Lisans
Türkçe
2012
Metalurji Mühendisliğiİstanbul Teknik ÜniversitesiMetalurji ve Malzeme Mühendisliği Ana Bilim Dalı
DOÇ. DR. FİLİZ ÇINAR ŞAHİN
- Mekanik alaşımlama ile CoCrFeMnNi ve AgCoCrFeNi esaslı yüksek entropi alaşım tozlarının sentezlenmesi ve karakterizasyonu
Synthesization of CoCrFeMnNi and AgCoCrFeNi high entropy alloy powders via mechanical alloying and their characterization
EBRU SARIOĞLU
Yüksek Lisans
Türkçe
2021
Mühendislik Bilimleriİstanbul Teknik ÜniversitesiMalzeme Bilimi ve Mühendisliği Ana Bilim Dalı
PROF. DR. BURAK ÖZKAL
- 1,4-diazabisiklo[2.2.2]oktan esaslı katyonik polimerlerin halka açılma metatez polimerizasyonu metodu ile sentezi ve antibakteriyel etkinliklerinin incelenmesi
Synthesis of 1,4-di̇azabi̇cyclo[2.2.2]octane based cationic polymers with ring opening metathesis polymerization technique and examination of their antibacterial activity
AYLİN KAYMAZ
- Synthesization of noble metal nanoparticles by pulsed laser ablation method in liquids and thin film applications
Soy metal nanooparçacıkların darbeli lazer ablasyon yöntemiyle sıvılarda üretimi ve ince film uygulamaları
HÜSEYİN AVNİ VURAL
Yüksek Lisans
İngilizce
2012
Fizik ve Fizik Mühendisliğiİhsan Doğramacı Bilkent ÜniversitesiMalzeme Bilimi ve Nanoteknoloji Ana Bilim Dalı
YRD. DOÇ. DR. BÜLEND ORTAÇ