Empirical Evaluation of Rectified Activations in Convolutional Network

Abstract

In this paper we investigate the performance of different types of rectified activation functions in convolutional neural network: standard rectified linear unit (ReLU), leaky rectified linear unit (Leaky ReLU), parametric rectified linear unit (PReLU) and a new randomized leaky rectified linear units (RReLU). We evaluate these activation function on standard image classification task. Our experiments suggest that incorporating a non-zero slope for negative part in rectified activation units could consistently improve the results. Thus our findings are negative on the common belief that sparsity is the key of good performance in ReLU. Moreover, on small scale dataset, using deterministic negative slope or learning it are both prone to overfitting. They are not as effective as using their randomized counterpart. By using RReLU, we achieved 75.68\% accuracy on CIFAR-100 test set without multiple test or ensemble.

References in corpus (4)

• Going Deeper with Convolutions
• Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
• Transferring Rich Feature Hierarchies for Robust Visual Tracking
• Deeply learned face representations are sparse, selective, and robust

Cited by in corpus (415)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Searching for Activation Functions
• Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
• Regularized Deep Networks in Intelligent Transportation Systems: A Taxonomy and a Case Study
• A survey on modern trainable activation functions
• DeepInf: Social Influence Prediction with Deep Learning
• Adversarial Attacks and Defences: A Survey
• Few-Shot Learning with Graph Neural Networks
• Recent Advances in Convolutional Neural Networks
• Systematic evaluation of CNN advances on the ImageNet
• Learning Tree-based Deep Model for Recommender Systems
• Deep learning electromagnetic inversion with convolutional neural networks
• SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud
• A Review of Object Detection Models based on Convolutional Neural Network
• Learning Image-adaptive 3D Lookup Tables for High Performance Photo Enhancement in Real-time
• GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs
• CAN: Creative Adversarial Networks, Generating "Art" by Learning About Styles and Deviating from Style Norms
• CardioXNet: A Novel Lightweight Deep Learning Framework for Cardiovascular Disease Classification Using Heart Sound Recordings
• High-Fidelity Generative Image Compression
• Real-Time Apple Detection System Using Embedded Systems With Hardware Accelerators: An Edge AI Application
• Regularization for Deep Learning: A Taxonomy
• Backflow Transformations via Neural Networks for Quantum Many-Body Wave-Functions
• Good Features to Correlate for Visual Tracking
• A Comprehensive Overview and Comparative Analysis on Deep Learning Models: CNN, RNN, LSTM, GRU
• Real-Time Radar-Based Gesture Detection and Recognition Built in an Edge-Computing Platform
• A Survey of FPGA-Based Neural Network Accelerator
• Exploring Spatial Significance via Hybrid Pyramidal Graph Network for Vehicle Re-identification
• Diffraction tomography with a deep image prior
• Scalable agent alignment via reward modeling: a research direction
• Review: Deep Learning in Electron Microscopy
• Terminal Brain Damage: Exposing the Graceless Degradation in Deep Neural Networks Under Hardware Fault Attacks
• A new method for faster and more accurate inference of species associations from big community data
• Building Damage Annotation on Post-Hurricane Satellite Imagery Based on Convolutional Neural Networks
• Conditional Variational Image Deraining
• Efficient Two-Stream Network for Violence Detection Using Separable Convolutional LSTM
• Compositional Fairness Constraints for Graph Embeddings
• ContraGAN: Contrastive Learning for Conditional Image Generation
• Segmentation of cell-level anomalies in electroluminescence images of photovoltaic modules
• Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameters
• Thermodynamically Consistent Machine-Learned Internal State Variable Approach for Data-Driven Modeling of Path-Dependent Materials
• QReLU and m-QReLU: Two novel quantum activation functions to aid medical diagnostics
• Optical Fringe Patterns Filtering Based on Multi-Stage Convolution Neural Network
• CASA-Based Speaker Identification Using Cascaded GMM-CNN Classifier in Noisy and Emotional Talking Conditions
• Deep neural network improves the estimation of polygenic risk scores for breast cancer
• A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks
• Probabilistic Neural Architecture Search
• Contrastive Learning of Structured World Models
• An ensemble deep learning technique for detecting suicidal ideation from posts in social media platforms
• Constructing Accurate and Efficient Deep Spiking Neural Networks with Double-threshold and Augmented Schemes
• Unsupervised Domain Adversarial Self-Calibration for Electromyographic-based Gesture Recognition
• Training Deep AutoEncoders for Collaborative Filtering
• BOIL: Towards Representation Change for Few-shot Learning
• DPFM: Deep Partial Functional Maps
• SiFall: Practical Online Fall Detection with RF Sensing
• Multi-Representational Learning for Offline Signature Verification using Multi-Loss Snapshot Ensemble of CNNs
• Universal Activation Function For Machine Learning
• GraphAIR: Graph Representation Learning with Neighborhood Aggregation and Interaction
• Neural Networks Fail to Learn Periodic Functions and How to Fix It
• On Transferability of Prompt Tuning for Natural Language Processing
• Average Biased ReLU Based CNN Descriptor for Improved Face Retrieval
• SRECG: ECG Signal Super-resolution Framework for Portable/Wearable Devices in Cardiac Arrhythmias Classification
• Lets keep it simple, Using simple architectures to outperform deeper and more complex architectures
• Adversarial Robustness of Deep Neural Networks: A Survey from a Formal Verification Perspective
• ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction
• Toward high-performance online HCCR: a CNN approach with DropDistortion, path signature and spatial stochastic max-pooling
• U2-Net: A Bayesian U-Net model with epistemic uncertainty feedback for photoreceptor layer segmentation in pathological OCT scans
• Sample Selection with Uncertainty of Losses for Learning with Noisy Labels
• Deep convolutional neural networks for multi-planar lung nodule detection: improvement in small nodule identification
• Time Series Classification from Scratch with Deep Neural Networks: A Strong Baseline
• Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection
• Activation Functions in Artificial Neural Networks: A Systematic Overview
• Revise Saturated Activation Functions
• A deep learning framework for the detection and quantification of drusen and reticular pseudodrusen on optical coherence tomography
• Speech Denoising Without Clean Training Data: A Noise2Noise Approach
• A robust single-pixel particle image velocimetry based on fully convolutional networks with cross-correlation embedded
• Only Train Once: A One-Shot Neural Network Training And Pruning Framework
• 3D Mesh Segmentation via Multi-branch 1D Convolutional Neural Networks
• Benchmarking Classic and Learned Navigation in Complex 3D Environments
• UNetGAN: A Robust Speech Enhancement Approach in Time Domain for Extremely Low Signal-to-noise Ratio Condition
• Extracting quantitative biological information from brightfield cell images using deep learning
• Efficiently utilizing complex-valued PolSAR image data via a multi-task deep learning framework
• Adaptive Image Sampling using Deep Learning and its Application on X-Ray Fluorescence Image Reconstruction
• ATFaceGAN: Single Face Image Restoration and Recognition from Atmospheric Turbulence
• Improving Efficiency in Convolutional Neural Network with Multilinear Filters
• Bridging the Gap Between Spectral and Spatial Domains in Graph Neural Networks
• End-to-End Learnable Multi-Scale Feature Compression for VCM
• secureTF: A Secure TensorFlow Framework
• Clustering and Unsupervised Anomaly Detection with L2 Normalized Deep Auto-Encoder Representations
• Exploring the likelihood of the 21-cm power spectrum with simulation-based inference
• Flatten-T Swish: a thresholded ReLU-Swish-like activation function for deep learning
• Acoustic scene classification using convolutional neural network and multiple-width frequency-delta data augmentation
• Deep speech inpainting of time-frequency masks
• Learning Graph Neural Networks with Positive and Unlabeled Nodes
• TensorSCONE: A Secure TensorFlow Framework using Intel SGX
• Learning Hierarchical Discourse-level Structure for Fake News Detection
• A Review of 1D Convolutional Neural Networks toward Unknown Substance Identification in Portable Raman Spectrometer
• Approximability of Discriminators Implies Diversity in GANs
• A Comprehensive and Modularized Statistical Framework for Gradient Norm Equality in Deep Neural Networks
• Non-Rigid Point Set Registration Networks
• NRC-Net: Automated noise robust cardio net for detecting valvular cardiac diseases using optimum transformation method with heart sound signals
• When Deep Learning Meets Multi-Task Learning in SAR ATR: Simultaneous Target Recognition and Segmentation
• TinyTL: Reduce Activations, Not Trainable Parameters for Efficient On-Device Learning
• A Self-Tuning Actor-Critic Algorithm
• Joint Optimization of Tree-based Index and Deep Model for Recommender Systems
• Adversarial Example in Remote Sensing Image Recognition
• ColorMapGAN: Unsupervised Domain Adaptation for Semantic Segmentation Using Color Mapping Generative Adversarial Networks
• Progression and Challenges of IoT in Healthcare: A Short Review
• Deep Transfer Learning with Graph Neural Network for Sensor-Based Human Activity Recognition
• Multivariate Time-series Anomaly Detection via Graph Attention Network
• Interpolated Convolutional Networks for 3D Point Cloud Understanding
• Improved Trainable Calibration Method for Neural Networks on Medical Imaging Classification
• FeederGAN: Synthetic Feeder Generation via Deep Graph Adversarial Nets
• Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learning
• Sample-Efficient and Surrogate-Based Design Optimization of Underwater Vehicle Hulls
• Universal Approximation with Deep Narrow Networks
• Unpaired Image Captioning via Scene Graph Alignments
• dCAM: Dimension-wise Class Activation Map for Explaining Multivariate Data Series Classification
• Geometric framework to predict structure from function in neural networks
• 3D Dense Face Alignment via Graph Convolution Networks
• Decentralized Learning of Generative Adversarial Networks from Non-iid Data
• ProbAct: A Probabilistic Activation Function for Deep Neural Networks
• Fault in your stars: An Analysis of Android App Reviews
• Latent-Domain Predictive Neural Speech Coding
• SURIMI: Supervised Radio Map Augmentation with Deep Learning and a Generative Adversarial Network for Fingerprint-based Indoor Positioning
• ConFoc: Content-Focus Protection Against Trojan Attacks on Neural Networks
• DiscrimNet: Semi-Supervised Action Recognition from Videos using Generative Adversarial Networks
• Graph-adaptive Rectified Linear Unit for Graph Neural Networks
• Exploiting the ConvLSTM: Human Action Recognition using Raw Depth Video-Based Recurrent Neural Networks
• Deep Hyperspectral Prior: Denoising, Inpainting, Super-Resolution
• ADOP: Approximate Differentiable One-Pixel Point Rendering
• Image Clustering using an Augmented Generative Adversarial Network and Information Maximization
• Likelihood-free Inference with Mixture Density Network
• DeepFlash: Turning a Flash Selfie into a Studio Portrait
• Fast and Provable ADMM for Learning with Generative Priors
• Memorization in Overparameterized Autoencoders
• Multimodal Anomaly Detection based on Deep Auto-Encoder for Object Slip Perception of Mobile Manipulation Robots
• CariGAN: Caricature Generation through Weakly Paired Adversarial Learning
• Coherent Point Drift Networks: Unsupervised Learning of Non-Rigid Point Set Registration
• A neural network emulator for the Lyman-$\alpha$ 1D flux power spectrum
• Detection of Accounting Anomalies in the Latent Space using Adversarial Autoencoder Neural Networks
• GMM-UNIT: Unsupervised Multi-Domain and Multi-Modal Image-to-Image Translation via Attribute Gaussian Mixture Modeling
• Physics-guided Convolutional Neural Network (PhyCNN) for Data-driven Seismic Response Modeling
• SRNet: Improving Generalization in 3D Human Pose Estimation with a Split-and-Recombine Approach
• Deep Learning in Searching the Spectroscopic Redshift of Quasars
• Adma-GAN: Attribute-Driven Memory Augmented GANs for Text-to-Image Generation
• Gesture-to-Gesture Translation in the Wild via Category-Independent Conditional Maps
• Synthesizing lesions using contextual GANs improves breast cancer classification on mammograms
• Listening while Speaking: Speech Chain by Deep Learning
• Improving prostate whole gland segmentation in t2-weighted MRI with synthetically generated data
• CorGAN: Correlation-Capturing Convolutional Generative Adversarial Networks for Generating Synthetic Healthcare Records
• Soft-Root-Sign Activation Function
• Evidence Networks: simple losses for fast, amortized, neural Bayesian model comparison
• EraseReLU: A Simple Way to Ease the Training of Deep Convolution Neural Networks
• Funnel Activation for Visual Recognition
• Transforming task representations to perform novel tasks
• DeepCancer: Detecting Cancer through Gene Expressions via Deep Generative Learning
• AReLU: Attention-based Rectified Linear Unit
• Accelerating Bayesian microseismic event location with deep learning
• Obstacle Avoidance Using a Monocular Camera
• Multimodal Urban Sound Tagging with Spatiotemporal Context
• Temporal Knowledge Graph Reasoning Based on Evolutional Representation Learning
• Deblending Galaxies with Generative Adversarial Networks
• Exploring the Design Space of Deep Convolutional Neural Networks at Large Scale
• CKConv: Continuous Kernel Convolution For Sequential Data
• Efficient Convolutional Neural Network Training with Direct Feedback Alignment
• Effects of the Nonlinearity in Activation Functions on the Performance of Deep Learning Models
• Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial Networks
• Improving Video Generation for Multi-functional Applications
• Machine Speech Chain with One-shot Speaker Adaptation
• Reinforcement Learning with A* and a Deep Heuristic
• Online and Offline Handwritten Chinese Character Recognition: A Comprehensive Study and New Benchmark
• Efficient Visual Computing with Camera RAW Snapshots
• Active Convolution: Learning the Shape of Convolution for Image Classification
• Deep Learning for Ultrasound Speed-of-Sound Reconstruction: Impacts of Training Data Diversity on Stability and Robustness
• TGAN: Deep Tensor Generative Adversarial Nets for Large Image Generation
• A likelihood approach to nonparametric estimation of a singular distribution using deep generative models
• Improving Description-based Person Re-identification by Multi-granularity Image-text Alignments
• Next2You: Robust Copresence Detection Based on Channel State Information
• Timing and characterization of shaped pulses with MHz ADCs in a detector system: a comparative study and deep learning approach
• Multiresolution Convolutional Autoencoders
• Pulsar Candidate Identification with Artificial Intelligence Techniques
• Unsupervised Learning of 3D Point Set Registration
• Abstractive Text Classification Using Sequence-to-convolution Neural Networks
• Deep Reasoning Networks: Thinking Fast and Slow
• DANet: Enhancing Small Object Detection through an Efficient Deformable Attention Network
• STMARL: A Spatio-Temporal Multi-Agent Reinforcement Learning Approach for Cooperative Traffic Light Control
• Improving Deep Neural Network with Multiple Parametric Exponential Linear Units
• Language Independent Single Document Image Super-Resolution using CNN for improved recognition
• Vehicle Re-identification Using Quadruple Directional Deep Learning Features
• Contextual Dropout: An Efficient Sample-Dependent Dropout Module
• Efficient end-to-end learning for quantizable representations
• Unsupervised 3D Reconstruction from a Single Image via Adversarial Learning
• Deep Density-aware Count Regressor
• Activate or Not: Learning Customized Activation
• Convolution-Weight-Distribution Assumption: Rethinking the Criteria of Channel Pruning
• Latent Variable Modelling with Hyperbolic Normalizing Flows
• Deep CT to MR Synthesis using Paired and Unpaired Data
• The PAU Survey: Background light estimation with deep learning techniques
• Activation Function Optimization Scheme for Image Classification
• End-to-end depth from motion with stabilized monocular videos
• Do biological constraints impair dendritic computation?
• Facial Expression Video Generation Based-On Spatio-temporal Convolutional GAN: FEV-GAN
• Self6D: Self-Supervised Monocular 6D Object Pose Estimation
• Neural Processes Mixed-Effect Models for Deep Normative Modeling of Clinical Neuroimaging Data
• Contributing Components of Metabolic Energy Models to Metabolic Cost Estimations in Gait
• DeepClimGAN: A High-Resolution Climate Data Generator
• Symbolic Network: Generalized Neural Policies for Relational MDPs
• Jointly embedding the local and global relations of heterogeneous graph for rumor detection
• Lightweight Convolutional Representations for On-Device Natural Language Processing
• Universal uncertainty estimation for nuclear detector signals with neural networks and ensemble learning
• Training Neural Networks by Using Power Linear Units (PoLUs)
• NeXtQSM -- A complete deep learning pipeline for data-consistent quantitative susceptibility mapping trained with hybrid data
• Progressive Point Cloud Deconvolution Generation Network
• Describe What to Change: A Text-guided Unsupervised Image-to-Image Translation Approach
• Deep Feature Fusion Network for Answer Quality Prediction in Community Question Answering
• A Deep Neuro-Fuzzy Network for Image Classification
• Learning Humanoid Robot Motions Through Deep Neural Networks
• Unsupervised Multi-Source Domain Adaptation for Person Re-Identification
• Knowledge-based Radiation Treatment Planning: A Data-driven Method Survey
• Topology-Aware Graph Pooling Networks
• You Only Look One-level Feature
• MITNet: GAN Enhanced Magnetic Induction Tomography Based on Complex CNN
• CIFAR-10: KNN-based Ensemble of Classifiers
• Exploratory State Representation Learning
• Deep Generative Model for Image Inpainting with Local Binary Pattern Learning and Spatial Attention
• Iterative peak-fitting of frequency-domain data via deep convolution neural networks
• Training Deep Neural Networks with Joint Quantization and Pruning of Weights and Activations
• Research on Mosaic Image Data Enhancement for Overlapping Ship Targets
• IVUS-Net: An Intravascular Ultrasound Segmentation Network
• Label-Noise Robust Generative Adversarial Networks
• Growing Cosine Unit: A Novel Oscillatory Activation Function That Can Speedup Training and Reduce Parameters in Convolutional Neural Networks
• Joint Learning of Graph Representation and Node Features in Graph Convolutional Neural Networks
• Few-shot Open-set Recognition by Transformation Consistency
• CPGAN: Full-Spectrum Content-Parsing Generative Adversarial Networks for Text-to-Image Synthesis
• Physical model simulator-trained neural network for computational 3D phase imaging of multiple-scattering samples
• Learning Global and Local Consistent Representations for Unsupervised Image Retrieval via Deep Graph Diffusion Networks
• Privacy Leakage of SIFT Features via Deep Generative Model based Image Reconstruction
• Addressing Fairness in Classification with a Model-Agnostic Multi-Objective Algorithm
• Map-level baryonification: unified treatment of weak lensing two-point and higher-order statistics
• Conditional WGANs with Adaptive Gradient Balancing for Sparse MRI Reconstruction
• Heterogeneous Graph Attention Networks for Early Detection of Rumors on Twitter
• Improving Ultrasound Tongue Image Reconstruction from Lip Images Using Self-supervised Learning and Attention Mechanism
• Increasing Depth Leads to U-Shaped Test Risk in Over-parameterized Convolutional Networks
• Automated Polysomnography Analysis for Detection of Non-Apneic and Non-Hypopneic Arousals using Feature Engineering and a Bidirectional LSTM Network
• CNN depth analysis with different channel inputs for Acoustic Scene Classification
• Deep Learning in Robotics: A Review of Recent Research
• Drop-Activation: Implicit Parameter Reduction and Harmonic Regularization
• 3D-to-2D Distillation for Indoor Scene Parsing
• SLA${}^2$P: Self-supervised Anomaly Detection with Adversarial Perturbation
• Relation Transformer Network
• On Detecting GANs and Retouching based Synthetic Alterations
• Accurate Calibration of Agent-based Epidemiological Models with Neural Network Surrogates
• Deciphering quantum fingerprints in electric conductance
• Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvature
• Cross-Vendor CT Image Data Harmonization Using CVH-CT
• Deep Motion Blur Removal Using Noisy/Blurry Image Pairs
• Gradient Information Guided Deraining with A Novel Network and Adversarial Training
• TanhSoft -- a family of activation functions combining Tanh and Softplus
• NAS-DIP: Learning Deep Image Prior with Neural Architecture Search
• ByteSGAN: A Semi-supervised Generative Adversarial Network for Encrypted Traffic Classification of SDN Edge Gateway in Green Communication Network
• Skin disease diagnosis with deep learning: a review
• SB-MTL: Score-based Meta Transfer-Learning for Cross-Domain Few-Shot Learning
• A High-Performance Adaptive Quantization Approach for Edge CNN Applications
• Hybrid summary statistics: neural weak lensing inference beyond the power spectrum
• Empirical Loss Landscape Analysis of Neural Network Activation Functions
• Towards Single-phase Single-stage Detection of Pulmonary Nodules in Chest CT Imaging
• Towards Non-saturating Recurrent Units for Modelling Long-term Dependencies
• On Reducing Negative Jacobian Determinant of the Deformation Predicted by Deep Registration Networks
• Natural-Logarithm-Rectified Activation Function in Convolutional Neural Networks
• Equivariant Entity-Relationship Networks
• Gaussian Process Neurons Learn Stochastic Activation Functions
• ARiA: Utilizing Richard's Curve for Controlling the Non-monotonicity of the Activation Function in Deep Neural Nets
• Identifying Most Walkable Direction for Navigation in an Outdoor Environment
• Semi-Supervised Learning Enabled by Multiscale Deep Neural Network Inversion
• Counterfactual Evaluation of Treatment Assignment Functions with Networked Observational Data
• Dynamic Neural Diversification: Path to Computationally Sustainable Neural Networks
• Piecewise Linear Units Improve Deep Neural Networks
• Amortized Bayesian inference for clustering models
• A Self-supervised Approach for Adversarial Robustness
• DeepFault: Fault Localization for Deep Neural Networks
• Lossless Compression of Mosaic Images with Convolutional Neural Network Prediction
• Sisyphus: A Cautionary Tale of Using Low-Degree Polynomial Activations in Privacy-Preserving Deep Learning
• Fixed smooth convolutional layer for avoiding checkerboard artifacts in CNNs
• Scenes and Surroundings: Scene Graph Generation using Relation Transformer
• Rapid quantification of COVID-19 pneumonia burden from computed tomography with convolutional LSTM networks
• STAN-CT: Standardizing CT Image using Generative Adversarial Network
• Deep-3DAligner: Unsupervised 3D Point Set Registration Network With Optimizable Latent Vector
• Advanced Graph-Based Deep Learning for Probabilistic Type Inference
• Autonomous Navigation in Dynamic Environments: Deep Learning-Based Approach
• Speech-to-speech Translation between Untranscribed Unknown Languages
• Emotion Generation and Recognition: A StarGAN Approach
• Content-Based Features to Rank Influential Hidden Services of the Tor Darknet
• Activation functions are not needed: the ratio net
• Conditioning Latent-Space Clusters for Real-World Anomaly Classification
• Federated Generative Adversarial Learning
• Key Information Extraction From Documents: Evaluation And Generator
• Coupled Gradient Estimators for Discrete Latent Variables
• GradNets: Dynamic Interpolation Between Neural Architectures
• Class-Distinct and Class-Mutual Image Generation with GANs
• Conditional Video Generation Using Action-Appearance Captions
• DeepLABNet: End-to-end Learning of Deep Radial Basis Networks with Fully Learnable Basis Functions
• CADA: Multi-scale Collaborative Adversarial Domain Adaptation for Unsupervised Optic Disc and Cup Segmentation
• Automated Let's Play Commentary
• Longitudinal Deep Kernel Gaussian Process Regression
• Decomposing Normal and Abnormal Features of Medical Images into Discrete Latent Codes for Content-Based Image Retrieval
• Solving Poisson's Equation using Deep Learning in Particle Simulation of PN Junction
• Voice Activity Detection in presence of background noise using EEG
• High-resolution, yet statistically relevant, analysis of damage in DP steel using artificial intelligence
• Adversarial Feature Learning of Online Monitoring Data for Operational Risk Assessment in Distribution Networks
• Contextual Joint Factor Acoustic Embeddings
• Shapley Explanation Networks
• Optimizing Memory Placement using Evolutionary Graph Reinforcement Learning
• Capacity allocation through neural network layers
• 5D Light Field Synthesis from a Monocular Video
• All-In-One: Artificial Association Neural Networks
• Improving EEG based continuous speech recognition using GAN
• Measuring Performance of Generative Adversarial Networks on Devanagari Script
• Person image generation with semantic attention network for person re-identification
• Sequence-to-Sequence ASR Optimization via Reinforcement Learning
• Ward2ICU: A Vital Signs Dataset of Inpatients from the General Ward
• CRAB: Class Representation Attentive BERT for Hate Speech Identification in Social Media
• Touch-based Curiosity for Sparse-Reward Tasks
• Question Directed Graph Attention Network for Numerical Reasoning over Text
• Radar Aided mmWave Vehicle-to-InfrastructureLink Configuration Using Deep Learning
• Toward Interactive Modulation for Photo-Realistic Image Restoration
• Sparse Attention with Linear Units
• LiStereo: Generate Dense Depth Maps from LIDAR and Stereo Imagery
• Subquadratic Overparameterization for Shallow Neural Networks
• Don't Fear Peculiar Activation Functions: EUAF and Beyond
• Comparisons among different stochastic selection of activation layers for convolutional neural networks for healthcare
• NICER: Aesthetic Image Enhancement with Humans in the Loop
• Battery Model Calibration with Deep Reinforcement Learning
• 3D Human Pose Estimation Based on 2D-3D Consistency with Synchronized Adversarial Training
• Applying Generative Adversarial Networks to Intelligent Subsurface Imaging and Identification
• Causal World Models by Unsupervised Deconfounding of Physical Dynamics
• Fractional moment-preserving initialization schemes for training deep neural networks
• Unsupervised Despeckling
• Contrastive Video-Language Segmentation
• Time-Distributed Feature Learning in Network Traffic Classification for Internet of Things
• Multi-scale Alignment and Contextual History for Attention Mechanism in Sequence-to-sequence Model
• Learning Sentence Embeddings for Coherence Modelling and Beyond
• Compact Deep Neural Networks for Computationally Efficient Gesture Classification From Electromyography Signals
• A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks
• Cost-Sensitive Deep Learning with Layer-Wise Cost Estimation
• On-Demand Learning for Deep Image Restoration
• Deep disentangled representations for volumetric reconstruction
• TGG: Transferable Graph Generation for Zero-shot and Few-shot Learning
• Using Non-Linear Causal Models to Study Aerosol-Cloud Interactions in the Southeast Pacific
• Optimal Transport Based Generative Autoencoders
• All-optical neural network quantum state tomography
• Social Influence Prediction with Train and Test Time Augmentation for Graph Neural Networks
• CounterNet: End-to-End Training of Prediction Aware Counterfactual Explanations
• Differentiable Set Operations for Algebraic Expressions
• Self-Supervised Graph Learning with Hyperbolic Embedding for Temporal Health Event Prediction
• Joint Learning of Portrait Intrinsic Decomposition and Relighting
• Unsupervised Anomaly Detection in MR Images using Multi-Contrast Information
• An End-to-End Attack on Text-based CAPTCHAs Based on Cycle-Consistent Generative Adversarial Network
• RBUE: A ReLU-Based Uncertainty Estimation Method of Deep Neural Networks
• Refining Semantic Segmentation with Superpixel by Transparent Initialization and Sparse Encoder
• Online trajectory recovery from offline handwritten Japanese kanji characters
• Multi-Metric Optimization using Generative Adversarial Networks for Near-End Speech Intelligibility Enhancement
• Learning Camera-Aware Noise Models
• End-to-End Efficient Representation Learning via Cascading Combinatorial Optimization
• GAN You Do the GAN GAN?
• An Adversarial Learning Based Approach for Unknown View Tomographic Reconstruction
• Introducing the DOME Activation Functions
• Graph Representation Learning for Spatial Image Steganalysis
• RefineGAN: Universally Generating Waveform Better than Ground Truth with Highly Accurate Pitch and Intensity Responses
• Hybrid BYOL-ViT: Efficient approach to deal with small datasets
• EffCNet: An Efficient CondenseNet for Image Classification on NXP BlueBox
• Goldilocks Neural Networks
• Implicit Quantile Networks For Emulation in Jet Physics
• Neural Networks for the Analysis of Traced Particles in Kinetic Plasma Simulations
• A Novel Information-Theoretic Objective to Disentangle Representations for Fair Classification
• Real-Time Object Detection and Localization in Compressive Sensed Video on Embedded Hardware
• A Transferable Adaptive Domain Adversarial Neural Network for Virtual Reality Augmented EMG-Based Gesture Recognition
• End-To-End Trainable Video Super-Resolution Based on a New Mechanism for Implicit Motion Estimation and Compensation
• Single Image Depth Prediction with Wavelet Decomposition
• Noise Robust Generative Adversarial Networks
• State Representation Learning from Demonstration
• Development of Quantized DNN Library for Exact Hardware Emulation
• Unsupervised Learning of Depth and Depth-of-Field Effect from Natural Images with Aperture Rendering Generative Adversarial Networks
• A Mechanism for Producing Aligned Latent Spaces with Autoencoders
• A new smart-cropping pipeline for prostate segmentation using deep learning networks
• Semi-Supervised Self-Growing Generative Adversarial Networks for Image Recognition
• Reconstruction of the Density Power Spectrum from Quasar Spectra using Machine Learning
• #StayHome or #Marathon? Social Media Enhanced Pandemic Surveillance on Spatial-temporal Dynamic Graphs
• Improving Multimodal fusion via Mutual Dependency Maximisation
• A Semi-Supervised Approach for Automatic Crystal Structure Classification
• Two-stream Spatiotemporal Feature for Video QA Task
• Nearly Minimal Over-Parametrization of Shallow Neural Networks
• Crowding in humans is unlike that in convolutional neural networks
• Relation3DMOT: Exploiting Deep Affinity for 3D Multi-Object Tracking from View Aggregation
• Discriminative Embedding Autoencoder with a Regressor Feedback for Zero-Shot Learning
• Generalized Categorisation of Digital Pathology Whole Image Slides using Unsupervised Learning
• An Empirical Study towards Understanding How Deep Convolutional Nets Recognize Falls
• Context-Aware Image Denoising with Auto-Threshold Canny Edge Detection to Suppress Adversarial Perturbation
• UVTomo-GAN: An adversarial learning based approach for unknown view X-ray tomographic reconstruction
• MSR-GAN: Multi-Segment Reconstruction via Adversarial Learning
• Continual Density Ratio Estimation in an Online Setting
• Structure-Aware Face Clustering on a Large-Scale Graph with $10^7$ Nodes
• Spatial Content Alignment For Pose Transfer
• Choice-Aware User Engagement Modeling andOptimization on Social Media
• Using holistic event information in the trigger
• Shuffle-Then-Assemble: Learning Object-Agnostic Visual Relationship Features
• Print Error Detection using Convolutional Neural Networks
• Neural network based limiter with transfer learning
• PAMTRI: Pose-Aware Multi-Task Learning for Vehicle Re-Identification Using Highly Randomized Synthetic Data
• Diffusion tensor driven image registration: a deep learning approach
• Quantifying the effect of representations on task complexity
• A temporal-to-spatial deep convolutional neural network for classification of hand movements from multichannel electromyography data
• Exploiting Temporal Attention Features for Effective Denoising in Videos
• Unsupervised Monocular Depth Estimation for Night-time Images using Adversarial Domain Feature Adaptation
• EIS -- a family of activation functions combining Exponential, ISRU, and Softplus
• LEGAN: Disentangled Manipulation of Directional Lighting and Facial Expressions by Leveraging Human Perceptual Judgements