Recovering Geometric Information with Learned Texture Perturbations
arXiv:2001.07253
Abstract
Regularization is used to avoid overfitting when training a neural network; unfortunately, this reduces the attainable level of detail hindering the ability to capture high-frequency information present in the training data. Even though various approaches may be used to re-introduce high-frequency detail, it typically does not match the training data and is often not time coherent. In the case of network inferred cloth, these sentiments manifest themselves via either a lack of detailed wrinkles or unnaturally appearing and/or time incoherent surrogate wrinkles. Thus, we propose a general strategy whereby high-frequency information is procedurally embedded into low-frequency data so that when the latter is smeared out by the network the former still retains its high-frequency detail. We illustrate this approach by learning texture coordinates which when smeared do not in turn smear out the high-frequency detail in the texture itself but merely smoothly distort it. Notably, we prescribe perturbed texture coordinates that are subsequently used to correct the over-smoothed appearance of inferred cloth, and correcting the appearance from multiple camera views naturally recovers lost geometric information.
References in corpus (14)
- Adam: A Method for Stochastic Optimization
- Conditional Generative Adversarial Nets
- Learning shape correspondence with anisotropic convolutional neural networks
- DeepWrinkles: Accurate and Realistic Clothing Modeling
- Coercing Machine Learning to Output Physically Accurate Results
- Geometric deep learning on graphs and manifolds using mixture model CNNs
- Video Based Reconstruction of 3D People Models
- Detailed, accurate, human shape estimation from clothed 3D scan sequences
- Characterizing and Improving Stability in Neural Style Transfer
- A Pixel-Based Framework for Data-Driven Clothing
- Learning to Reconstruct People in Clothing from a Single RGB Camera
- A Style-Aware Content Loss for Real-time HD Style Transfer
- Detailed Human Avatars from Monocular Video
- Variational Autoencoders for Deforming 3D Mesh Models