Soft Autoencoder and Its Wavelet Adaptation Interpretation
arXiv:1812.11675
Abstract
Recently, deep learning becomes the main focus of machine learning research and has greatly impacted many important fields. However, deep learning is criticized for lack of interpretability. As a successful unsupervised model in deep learning, the autoencoder embraces a wide spectrum of applications, yet it suffers from the model opaqueness as well. In this paper, we propose a new type of convolutional autoencoders, termed as Soft Autoencoder (Soft-AE), in which the activation functions of encoding layers are implemented with adaptable soft-thresholding units while decoding layers are realized with linear units. Consequently, Soft-AE can be naturally interpreted as a learned cascaded wavelet shrinkage system. Our denoising experiments demonstrate that Soft-AE not only is interpretable but also offers a competitive performance relative to its counterparts. Furthermore, we propose a generalized linear unit (GenLU) to make an autoencoder more adaptive in nonlinearly filtering images and data, such as denoising and deblurring.
arXiv commentThis manuscript is out-of-date
References in corpus (7)
- Gaussian Error Linear Units (GELUs)
- Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)
- Understanding Black-box Predictions via Influence Functions
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
- Stacked What-Where Auto-encoders
- k-Sparse Autoencoders
- A Geometric View of Optimal Transportation and Generative Model