ADADELTA: An Adaptive Learning Rate Method

Abstract

We present a novel per-dimension learning rate method for gradient descent called ADADELTA. The method dynamically adapts over time using only first order information and has minimal computational overhead beyond vanilla stochastic gradient descent. The method requires no manual tuning of a learning rate and appears robust to noisy gradient information, different model architecture choices, various data modalities and selection of hyperparameters. We show promising results compared to other methods on the MNIST digit classification task using a single machine and on a large scale voice dataset in a distributed cluster environment.

6 pages

Cited by in corpus (1218)

• Adam: A Method for Stochastic Optimization
• Deep Learning in Neural Networks: An Overview
• Neural Machine Translation by Jointly Learning to Align and Translate
• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• An overview of gradient descent optimization algorithms
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Deep learning for time series classification: a review
• In Defense of the Triplet Loss for Person Re-Identification
• A Critical Review of Recurrent Neural Networks for Sequence Learning
• Attention-Based Models for Speech Recognition
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Asynchronous Methods for Deep Reinforcement Learning
• Machine Learning Force Fields
• VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera
• Convolutional Neural Networks for Sentence Classification
• A Comparative Study on Transformer vs RNN in Speech Applications
• An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networks
• Applications of deep learning in stock market prediction: recent progress
• Deep Recurrent Q-Learning for Partially Observable MDPs
• On the Variance of the Adaptive Learning Rate and Beyond
• Dual Learning for Machine Translation
• Focusing Attention: Towards Accurate Text Recognition in Natural Images
• Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
• AtomNet: A Deep Convolutional Neural Network for Bioactivity Prediction in Structure-based Drug Discovery
• Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images
• On Using Monolingual Corpora in Neural Machine Translation
• Beyond Sharing Weights for Deep Domain Adaptation
• Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification
• Hyper-Parameter Optimization: A Review of Algorithms and Applications
• A large annotated corpus for learning natural language inference
• Optimizing Neural Networks with Kronecker-factored Approximate Curvature
• Gabor Convolutional Networks
• Jointly Multiple Events Extraction via Attention-based Graph Information Aggregation
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
• Deep Learning is Robust to Massive Label Noise
• Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
• Focal onset seizure prediction using convolutional networks
• Scalable Variational Gaussian Process Classification
• Evaluation of Session-based Recommendation Algorithms
• A Comprehensive Analysis of Deep Regression
• Distilling Task-Specific Knowledge from BERT into Simple Neural Networks
• Gravity Spy: Integrating Advanced LIGO Detector Characterization, Machine Learning, and Citizen Science
• MADE: Masked Autoencoder for Distribution Estimation
• Efficient training of physics-informed neural networks via importance sampling
• Recent Advances in Convolutional Neural Networks
• Neural Rating Regression with Abstractive Tips Generation for Recommendation
• Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling
• Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks
• ReasoNet: Learning to Stop Reading in Machine Comprehension
• ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering
• Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks
• Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder
• A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects
• Adding Gradient Noise Improves Learning for Very Deep Networks
• CMA-ES for Hyperparameter Optimization of Deep Neural Networks
• Learning Representations of Ultrahigh-dimensional Data for Random Distance-based Outlier Detection
• Natural Language Inference over Interaction Space
• RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism
• A Comparison of Optimization Algorithms for Deep Learning
• Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead
• Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning
• End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
• ECG arrhythmia classification using a 2-D convolutional neural network
• Neural Optimizer Search with Reinforcement Learning
• Learning Stochastic Recurrent Networks
• Harnessing Deep Neural Networks with Logic Rules
• The data synergy effects of time-series deep learning models in hydrology
• Describing Videos by Exploiting Temporal Structure
• Deep Neural Network Compression for Aircraft Collision Avoidance Systems
• A Comparative Study of Fruit Detection and Counting Methods for Yield Mapping in Apple Orchards
• A Comprehensive Study of Deep Bidirectional LSTM RNNs for Acoustic Modeling in Speech Recognition
• Efficient Character-level Document Classification by Combining Convolution and Recurrent Layers
• Question Answering and Question Generation as Dual Tasks
• Online Batch Selection for Faster Training of Neural Networks
• Simulator-free Solution of High-Dimensional Stochastic Elliptic Partial Differential Equations using Deep Neural Networks
• How Does Learning Rate Decay Help Modern Neural Networks?
• Adafactor: Adaptive Learning Rates with Sublinear Memory Cost
• GHRS: Graph-based Hybrid Recommendation System with Application to Movie Recommendation
• Identification of heavy, energetic, hadronically decaying particles using machine-learning techniques
• SMOTified-GAN for class imbalanced pattern classification problems
• Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks
• Automated Machine Learning: State-of-The-Art and Open Challenges
• Transformers with convolutional context for ASR
• Is Neural Machine Translation Ready for Deployment? A Case Study on 30 Translation Directions
• Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement
• Database Meets Deep Learning: Challenges and Opportunities
• Equilibrated adaptive learning rates for non-convex optimization
• Delving Deeper into Convolutional Networks for Learning Video Representations
• Riemannian Adaptive Optimization Methods
• Deep Shading: Convolutional Neural Networks for Screen-Space Shading
• Discovering Hidden Factors of Variation in Deep Networks
• Finding strong lenses in CFHTLS using convolutional neural networks
• Knowledge Matters: Importance of Prior Information for Optimization
• Compression-aware Training of Deep Networks
• Optimization for deep learning: theory and algorithms
• A Hierarchical Structured Self-Attentive Model for Extractive Document Summarization (HSSAS)
• Learning to detect chest radiographs containing lung nodules using visual attention networks
• Can neural machine translation do simultaneous translation?
• Deep Learning in Mobile and Wireless Networking: A Survey
• On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization
• AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients
• End-to-end LSTM-based dialog control optimized with supervised and reinforcement learning
• A Deep Network Model for Paraphrase Detection in Short Text Messages
• The power of deeper networks for expressing natural functions
• Medical Concept Representation Learning from Electronic Health Records and its Application on Heart Failure Prediction
• DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding
• CNN+RNN Depth and Skeleton based Dynamic Hand Gesture Recognition
• Style Transfer as Unsupervised Machine Translation
• Transfer Learning across Low-Resource, Related Languages for Neural Machine Translation
• Recent Advances in Object Detection in the Age of Deep Convolutional Neural Networks
• Continental-scale streamflow modeling of basins with reservoirs: towards a coherent deep-learning-based strategy
• Black box variational inference for state space models
• Variants of RMSProp and Adagrad with Logarithmic Regret Bounds
• Protein Secondary Structure Prediction with Long Short Term Memory Networks
• Obesity Prediction with EHR Data: A deep learning approach with interpretable elements
• What Do We Understand About Convolutional Networks?
• Latent Predictor Networks for Code Generation
• Robust Unsupervised Domain Adaptation for Neural Networks via Moment Alignment
• GRAM: Graph-based Attention Model for Healthcare Representation Learning
• Deep Learning for Medical Image Segmentation
• Physics and Equality Constrained Artificial Neural Networks: Application to Forward and Inverse Problems with Multi-fidelity Data Fusion
• Whetstone: A Method for Training Deep Artificial Neural Networks for Binary Communication
• Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial
• Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
• On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization
• Character-level and Multi-channel Convolutional Neural Networks for Large-scale Authorship Attribution
• On the Origin of Deep Learning
• Physics-Driven Regularization of Deep Neural Networks for Enhanced Engineering Design and Analysis
• THUMT: An Open Source Toolkit for Neural Machine Translation
• Two are Better than One: An Ensemble of Retrieval- and Generation-Based Dialog Systems
• Robust Scene Text Recognition with Automatic Rectification
• Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks
• Evaluating Differentially Private Machine Learning in Practice
• Convolutional neural network identification of galaxy post-mergers in UNIONS using IllustrisTNG
• Classify or Select: Neural Architectures for Extractive Document Summarization
• Deep Learning Methods for Solving Linear Inverse Problems: Research Directions and Paradigms
• Human Activity Recognition Based on Wearable Sensor Data: A Standardization of the State-of-the-Art
• Tutorial: How to Train a Neural Network Potential
• Combining Neural Networks and Log-linear Models to Improve Relation Extraction
• Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients
• On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization
• Hierarchical Attention Network for Action Recognition in Videos
• Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling
• Dive into Deep Learning
• Focal Visual-Text Attention for Visual Question Answering
• Dependency-based Convolutional Neural Networks for Sentence Embedding
• Chinese Song Iambics Generation with Neural Attention-based Model
• An Incremental Dimensionality Reduction Method for Visualizing Streaming Multidimensional Data
• What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis
• An analysis on the use of autoencoders for representation learning: fundamentals, learning task case studies, explainability and challenges
• Missing Data Imputation for Supervised Learning
• GPU Asynchronous Stochastic Gradient Descent to Speed Up Neural Network Training
• Applications of Online Deep Learning for Crisis Response Using Social Media Information
• An Extensive Study on Cross-Dataset Bias and Evaluation Metrics Interpretation for Machine Learning applied to Gastrointestinal Tract Abnormality Classification
• Collaborative Filtering with Recurrent Neural Networks
• Chinese Poetry Generation with Planning based Neural Network
• Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response
• Improving Malaria Parasite Detection from Red Blood Cell using Deep Convolutional Neural Networks
• CAN-NER: Convolutional Attention Network for Chinese Named Entity Recognition
• MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension
• End-to-end spoofing detection with raw waveform CLDNNs
• Bump Hunting in Latent Space
• Attending to Characters in Neural Sequence Labeling Models
• An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition
• Low-Resource Named Entity Recognition with Cross-Lingual, Character-Level Neural Conditional Random Fields
• Online Hybrid CTC/Attention End-to-End Automatic Speech Recognition Architecture
• Bridging the Gap between Stochastic Gradient MCMC and Stochastic Optimization
• Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout
• Learning like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images
• Nostalgic Adam: Weighting more of the past gradients when designing the adaptive learning rate
• Character-Level Incremental Speech Recognition with Recurrent Neural Networks
• Distraction-Based Neural Networks for Document Summarization
• Convolutional Neural Networks with Gated Recurrent Connections
• Towards the Limit of Network Quantization
• Zero-Shot Visual Question Answering
• EDropout: Energy-Based Dropout and Pruning of Deep Neural Networks
• Towards Neural Network-based Reasoning
• Variational Bayes with Synthetic Likelihood
• Polyphonic Sound Event Detection by using Capsule Neural Networks
• Analysis and Optimization of Convolutional Neural Network Architectures
• Unsupervised, Efficient and Semantic Expertise Retrieval
• AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learning
• Morphological classification of galaxies with deep learning: comparing 3-way and 4-way CNNs
• Imposing Hard Constraints on Deep Networks: Promises and Limitations
• GPU-Accelerated Drug Discovery with Docking on the Summit Supercomputer: Porting, Optimization, and Application to COVID-19 Research
• WNGrad: Learn the Learning Rate in Gradient Descent
• Learning What Data to Learn
• How many preprints have actually been printed and why: a case study of computer science preprints on arXiv
• Temporal Attention Model for Neural Machine Translation
• Strong error analysis for stochastic gradient descent optimization algorithms
• Tree-structured composition in neural networks without tree-structured architectures
• Deep learning for plasma tomography using the bolometer system at JET
• Graph neural network initialisation of quantum approximate optimisation
• Adam revisited: a weighted past gradients perspective
• Deep Reinforcement Learning in Parameterized Action Space
• 2D Attentional Irregular Scene Text Recognizer
• Automated detection of vulnerable plaque in intravascular ultrasound images
• DFTerNet: Towards 2-bit Dynamic Fusion Networks for Accurate Human Activity Recognition
• Learning Gradient Descent: Better Generalization and Longer Horizons
• Neighborhood Mixture Model for Knowledge Base Completion
• Deep Multi-Task Learning for Malware Image Classification
• Limits of End-to-End Learning
• Unit Tests for Stochastic Optimization
• An Optimal Control Approach to Deep Learning and Applications to Discrete-Weight Neural Networks
• Time Series Classification from Scratch with Deep Neural Networks: A Strong Baseline
• Identifying Clickbait: A Multi-Strategy Approach Using Neural Networks
• Neural Sentence Ordering
• Noisy Parallel Approximate Decoding for Conditional Recurrent Language Model
• S-Net: From Answer Extraction to Answer Generation for Machine Reading Comprehension
• Quantitative Fine-Grained Human Evaluation of Machine Translation Systems: a Case Study on English to Croatian
• Unsupervised Visual Feature Learning with Spike-timing-dependent Plasticity: How Far are we from Traditional Feature Learning Approaches?
• Style-based quantum generative adversarial networks for Monte Carlo events
• Modeling Relational Information in Question-Answer Pairs with Convolutional Neural Networks
• Actor-Critic based Training Framework for Abstractive Summarization
• Deep Architectures for Automated Seizure Detection in Scalp EEGs
• Out-of-distribution Detection in Classifiers via Generation
• Neural Sentence Embedding using Only In-domain Sentences for Out-of-domain Sentence Detection in Dialog Systems
• Multi-talker ASR for an unknown number of sources: Joint training of source counting, separation and ASR
• Larger-Context Language Modelling
• Gradient Descent based Optimization Algorithms for Deep Learning Models Training
• A Survey of Optimization Methods from a Machine Learning Perspective
• High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks
• DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks
• Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer
• Brain Tumor Synthetic Segmentation in 3D Multimodal MRI Scans
• Alzheimer's Disease Diagnostics by a Deeply Supervised Adaptable 3D Convolutional Network
• End-to-end training of time domain audio separation and recognition
• Deep Reinforcement Learning of Cell Movement in the Early Stage of C. elegans Embryogenesis
• DopeLearning: A Computational Approach to Rap Lyrics Generation
• Linear dynamical neural population models through nonlinear embeddings
• Discriminative Acoustic Word Embeddings: Recurrent Neural Network-Based Approaches
• Reconstruction Network for Video Captioning
• Multi-task Prediction of Disease Onsets from Longitudinal Lab Tests
• Towards Binary-Valued Gates for Robust LSTM Training
• Rapid Classification of Crisis-Related Data on Social Networks using Convolutional Neural Networks
• Stochastic gradient variational Bayes for gamma approximating distributions
• Global Convergence of Adaptive Gradient Methods for An Over-parameterized Neural Network
• SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning
• Neural Probabilistic Model for Non-projective MST Parsing
• Learning an Adaptive Learning Rate Schedule
• Exploiting Convolutional Neural Network for Risk Prediction with Medical Feature Embedding
• Big Batch SGD: Automated Inference using Adaptive Batch Sizes
• Neural Headline Generation with Sentence-wise Optimization
• An empirical study on the effectiveness of images in Multimodal Neural Machine Translation
• Time-sensitive Learning for Heterogeneous Federated Edge Intelligence
• Context is Key: Grammatical Error Detection with Contextual Word Representations
• Transforming Cooling Optimization for Green Data Center via Deep Reinforcement Learning
• Adversarial Generation of Training Examples: Applications to Moving Vehicle License Plate Recognition
• Similarity Learning for Authorship Verification in Social Media
• A Comprehensive Overview and Survey of Recent Advances in Meta-Learning
• Hamiltonian Descent Methods
• Deep Latent Dirichlet Allocation with Topic-Layer-Adaptive Stochastic Gradient Riemannian MCMC
• Coercing Machine Learning to Output Physically Accurate Results
• An Improvement of Data Classification Using Random Multimodel Deep Learning (RMDL)
• Brain volume: An important determinant of functional outcome after acute ischemic stroke
• Hierarchical LSTM with Adjusted Temporal Attention for Video Captioning
• InMyFace: Inertial and Mechanomyography-Based Sensor Fusion for Wearable Facial Activity Recognition
• Recurrent Neural Networks with External Memory for Language Understanding
• Universal Adversarial Audio Perturbations
• Bayesian neural networks via MCMC: a Python-based tutorial
• Coverage Embedding Models for Neural Machine Translation
• Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers
• Deep Gamblers: Learning to Abstain with Portfolio Theory
• On Physics-Informed Neural Networks for Quantum Computers
• Edit Probability for Scene Text Recognition
• Oriented Response Networks
• Accelerating multimodal gravitational waveforms from precessing compact binaries with artificial neural networks
• LHC analysis-specific datasets with Generative Adversarial Networks
• A Tutorial on Deep Latent Variable Models of Natural Language
• Doubly Convolutional Neural Networks
• Automatic Three-Dimensional Cephalometric Annotation System Using Three-Dimensional Convolutional Neural Networks
• A Kronecker-factored approximate Fisher matrix for convolution layers
• Joint Training for Neural Machine Translation Models with Monolingual Data
• ænet-PyTorch: a GPU-supported implementation for machine learning atomic potentials training
• Learning Deep ResNet Blocks Sequentially using Boosting Theory
• Revisiting Fine-tuning for Few-shot Learning
• Towards Explainable and Controllable Open Domain Dialogue Generation with Dialogue Acts
• Latent Variable Dialogue Models and their Diversity
• Guided parallelized stochastic gradient descent for delay compensation
• Compositional Generalization by Learning Analytical Expressions
• TheanoLM - An Extensible Toolkit for Neural Network Language Modeling
• Supervised Attentions for Neural Machine Translation
• Bridging the Gap between Training and Inference for Neural Machine Translation
• Word Embeddings and Their Use In Sentence Classification Tasks
• Style Transfer in Text: Exploration and Evaluation
• Amortized Variational Inference: A Systematic Review
• First Result on Arabic Neural Machine Translation
• An Adaptive and Momental Bound Method for Stochastic Learning
• Laplacian Smoothing Gradient Descent
• AdaShift: Decorrelation and Convergence of Adaptive Learning Rate Methods
• Towards Understanding Label Smoothing
• Learning multi-faceted representations of individuals from heterogeneous evidence using neural networks
• A GRU-Gated Attention Model for Neural Machine Translation
• Robust Bi-Tempered Logistic Loss Based on Bregman Divergences
• AdaSwarm: Augmenting Gradient-Based optimizers in Deep Learning with Swarm Intelligence
• A Comparative Study on Machine Learning Algorithms for the Control of a Wall Following Robot
• Neural Machine Translation with Supervised Attention
• Neural Enquirer: Learning to Query Tables with Natural Language
• Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling
• Neural Emoji Recommendation in Dialogue Systems
• Deep Open-Set Recognition for Silicon Wafer Production Monitoring
• A Deep Memory-based Architecture for Sequence-to-Sequence Learning
• Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks
• A Framework for Parallel and Distributed Training of Neural Networks
• Online Learning for Neural Machine Translation Post-editing
• Convergent Block Coordinate Descent for Training Tikhonov Regularized Deep Neural Networks
• BlinkML: Efficient Maximum Likelihood Estimation with Probabilistic Guarantees
• Advances in Variational Inference
• DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentation
• Como funciona o Deep Learning
• Memory-enhanced Decoder for Neural Machine Translation
• Answer Sequence Learning with Neural Networks for Answer Selection in Community Question Answering
• AdaX: Adaptive Gradient Descent with Exponential Long Term Memory
• Fault in your stars: An Analysis of Android App Reviews
• Interactive Attention for Neural Machine Translation
• Inverse Transport Networks
• ATASI-Net: An Efficient Sparse Reconstruction Network for Tomographic SAR Imaging with Adaptive Threshold
• Coupling Adaptive Batch Sizes with Learning Rates
• Effectiveness of Distillation Attack and Countermeasure on Neural Network Watermarking
• Audio Visual Emotion Recognition with Temporal Alignment and Perception Attention
• Helping results assessment by adding explainable elements to the deep relevance matching model
• Hessian-Aware Zeroth-Order Optimization for Black-Box Adversarial Attack
• Dalek -- a deep-learning emulator for TARDIS
• Finding Influential Instances for Distantly Supervised Relation Extraction
• Deep learning with photosensor timing information as a background rejection method for the Cherenkov Telescope Array
• Deep Frank-Wolfe For Neural Network Optimization
• Riemannian adaptive stochastic gradient algorithms on matrix manifolds
• Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptron
• ElegansNet: a brief scientific report and initial experiments
• DeepStreet: A deep learning powered urban street network generation module
• Semantic Instance Annotation of Street Scenes by 3D to 2D Label Transfer
• Optimized Three Deep Learning Models Based-PSO Hyperparameters for Beijing PM2.5 Prediction
• LCSTS: A Large Scale Chinese Short Text Summarization Dataset
• Deep learning: Technical introduction
• Modeling Source Syntax for Neural Machine Translation
• Knowledge Representation Learning: A Quantitative Review
• Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates
• Attention-Based Multimodal Fusion for Video Description
• Rationale-Augmented Convolutional Neural Networks for Text Classification
• One Sentence One Model for Neural Machine Translation
• Gradient Descent Bit-Flipping Decoding with Momentum
• Video Captioning via Hierarchical Reinforcement Learning
• Analyzing analytical methods: The case of phonology in neural models of spoken language
• From Patch to Image Segmentation using Fully Convolutional Networks -- Application to Retinal Images
• Path optimization in 0+1 dimensional QCD at finite density
• Self-similarity Prior Distillation for Unsupervised Remote Physiological Measurement
• Iterative Neural Autoregressive Distribution Estimator (NADE-k)
• Sparse Multi-Output Gaussian Processes for Medical Time Series Prediction
• A practical tutorial on Variational Bayes
• Local AdaAlter: Communication-Efficient Stochastic Gradient Descent with Adaptive Learning Rates
• Inhibited Softmax for Uncertainty Estimation in Neural Networks
• Convolutional Color Constancy
• Advances in Joint CTC-Attention based End-to-End Speech Recognition with a Deep CNN Encoder and RNN-LM
• Gradient Descent: The Ultimate Optimizer
• Fast and Accurate Sparse Coding of Visual Stimuli with a Simple, Ultra-Low-Energy Spiking Architecture
• Toward automatic comparison of visualization techniques: Application to graph visualization
• Churn analysis using deep convolutional neural networks and autoencoders
• Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera
• MetaGrad: Multiple Learning Rates in Online Learning
• QA4IE: A Question Answering based Framework for Information Extraction
• Confidence through Attention
• Multi-Scale Attention with Dense Encoder for Handwritten Mathematical Expression Recognition
• Determining electronic properties from L-edge X-ray absorption spectra of transition metal compounds with artificial neural networks
• SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognition
• Basis Pursuit Denoising via Recurrent Neural Network Applied to Super-resolving SAR Tomography
• Disentangling Boosted Higgs Boson Production Modes with Machine Learning
• Comparing Rule-Based and Deep Learning Models for Patient Phenotyping
• The Devil Is in the Details: An Efficient Convolutional Neural Network for Transport Mode Detection
• Predicting Audience's Laughter Using Convolutional Neural Network
• Normalized Direction-preserving Adam
• KATE: K-Competitive Autoencoder for Text
• Particle Identification In Camera Image Sensors Using Computer Vision
• Modality to Modality Translation: An Adversarial Representation Learning and Graph Fusion Network for Multimodal Fusion
• Self-Attentive Residual Decoder for Neural Machine Translation
• Deep Complex Networks for Protocol-Agnostic Radio Frequency Device Fingerprinting in the Wild
• Stein variational gradient descent with local approximations
• Mixing ADAM and SGD: a Combined Optimization Method
• Robust Lexical Features for Improved Neural Network Named-Entity Recognition
• Generative Adversarial Networks as Variational Training of Energy Based Models
• Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning
• Character-level Convolutional Network for Text Classification Applied to Chinese Corpus
• Toward Mention Detection Robustness with Recurrent Neural Networks
• Memory-augmented Neural Machine Translation
• A Simplified Fully Quantized Transformer for End-to-end Speech Recognition
• Question Answering through Transfer Learning from Large Fine-grained Supervision Data
• Deep Robust Kalman Filter
• Does Adam optimizer keep close to the optimal point?
• Model error covariance estimation in particle and ensemble Kalman filters using an online expectation-maximization algorithm
• Improving Interpretability of Deep Neural Networks with Semantic Information
• Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation
• A Constrained Sequence-to-Sequence Neural Model for Sentence Simplification
• Hierarchical LSTMs with Adaptive Attention for Visual Captioning
• Gaussian variational approximation for high-dimensional state space models
• Feature learning for efficient ASR-free keyword spotting in low-resource languages
• CoolMomentum: A Method for Stochastic Optimization by Langevin Dynamics with Simulated Annealing
• Transfer Learning for Scene Text Recognition in Indian Languages
• Keep the Gradients Flowing: Using Gradient Flow to Study Sparse Network Optimization
• Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives
• Neural Net Models for Open-Domain Discourse Coherence
• Neural Networks Models for Entity Discovery and Linking
• Where Did My Optimum Go?: An Empirical Analysis of Gradient Descent Optimization in Policy Gradient Methods
• Siamese recurrent networks learn first-order logic reasoning and exhibit zero-shot compositional generalization
• From Deterministic to Generative: Multi-Modal Stochastic RNNs for Video Captioning
• Adaptive Gradient Descent without Descent
• Adversarial TableQA: Attention Supervision for Question Answering on Tables
• Latent Gaussian Processes for Distribution Estimation of Multivariate Categorical Data
• Deep-XFCT: Deep learning 3D-mineral liberation analysis with micro X-ray fluorescence and computed tomography
• DeepAM: Migrate APIs with Multi-modal Sequence to Sequence Learning
• Self-supervised Learning with Physics-aware Neural Networks I: Galaxy Model Fitting
• 1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed
• SimAug: Learning Robust Representations from Simulation for Trajectory Prediction
• Learning to Optimize in Model Predictive Control
• Reinforcement Learning for Learning Rate Control
• Triangular Architecture for Rare Language Translation
• Watch, Listen, and Describe: Globally and Locally Aligned Cross-Modal Attentions for Video Captioning
• A Rigorous Uncertainty-Aware Quantification Framework Is Essential for Reproducible and Replicable Machine Learning Workflows
• Functional connectivity patterns of autism spectrum disorder identified by deep feature learning
• Convolutional 3D to 2D Patch Conversion for Pixel-wise Glioma Segmentation in MRI Scans
• Estimation of discrete choice models with hybrid stochastic adaptive batch size algorithms
• Smarnet: Teaching Machines to Read and Comprehend Like Human
• Convergence of Newton-MR under Inexact Hessian Information
• On the effects of biased quantum random numbers on the initialization of artificial neural networks
• Image denoising and restoration with CNN-LSTM Encoder Decoder with Direct Attention
• Ruminating Reader: Reasoning with Gated Multi-Hop Attention
• Learning to Filter with Predictive State Inference Machines
• Denoising autoencoder with modulated lateral connections learns invariant representations of natural images
• Trainable Greedy Decoding for Neural Machine Translation
• Nematus: a Toolkit for Neural Machine Translation
• Eve: A Gradient Based Optimization Method with Locally and Globally Adaptive Learning Rates
• Are we Forgetting about Compositional Optimisers in Bayesian Optimisation?
• A Quantitative Comparison of Epistemic Uncertainty Maps Applied to Multi-Class Segmentation
• Activation Ensembles for Deep Neural Networks
• Demystifying Learning Rate Policies for High Accuracy Training of Deep Neural Networks
• A Deep-Learning Approach for Operation of an Automated Realtime Flare Forecast
• Distributed Learning of Deep Neural Networks using Independent Subnet Training
• Effective Federated Adaptive Gradient Methods with Non-IID Decentralized Data
• Deep Neural Machine Translation with Linear Associative Unit
• On Learning Rates and Schrödinger Operators
• Study and development of a Computer-Aided Diagnosis system for classification of chest x-ray images using convolutional neural networks pre-trained for ImageNet and data augmentation
• Tracking State Changes in Procedural Text: A Challenge Dataset and Models for Process Paragraph Comprehension
• Improving Classification Rate of Schizophrenia Using a Multimodal Multi-Layer Perceptron Model with Structural and Functional MR
• WeMix: How to Better Utilize Data Augmentation
• Continual Learning in Neural Networks
• Prediction of Kidney Function from Biopsy Images Using Convolutional Neural Networks
• Learning to Understand Phrases by Embedding the Dictionary
• First Steps Toward Incorporating Image Based Diagnostics Into Particle Accelerator Control Systems Using Convolutional Neural Networks
• Stochastic Nonconvex Optimization with Large Minibatches
• Memorized Sparse Backpropagation
• Knowledge-aware Attention Network for Protein-Protein Interaction Extraction
• Determinantal Point Processes for Mini-Batch Diversification
• Denoising without access to clean data using a partitioned autoencoder
• "When they say weed causes depression, but it's your fav antidepressant": Knowledge-aware Attention Framework for Relationship Extraction
• A Study of Deep Learning Robustness Against Computation Failures
• Gradient Amplification: An efficient way to train deep neural networks
• Improving Implicit Semantic Role Labeling by Predicting Semantic Frame Arguments
• Fair Comparison: Quantifying Variance in Resultsfor Fine-grained Visual Categorization
• Online and Offline Handwritten Chinese Character Recognition: A Comprehensive Study and New Benchmark
• Blitzkriging: Kronecker-structured Stochastic Gaussian Processes
• ADASECANT: Robust Adaptive Secant Method for Stochastic Gradient
• Learning to Remember Translation History with a Continuous Cache
• Scalable Planning with Tensorflow for Hybrid Nonlinear Domains
• Spectrum Correction: Acoustic Scene Classification with Mismatched Recording Devices
• A general system of differential equations to model first order adaptive algorithms
• Automated Spectral Kernel Learning
• A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performance
• Prior Knowledge Integration for Neural Machine Translation using Posterior Regularization
• Large-Batch Training for LSTM and Beyond
• UTRNet: High-Resolution Urdu Text Recognition In Printed Documents
• Kissing Cuisines: Exploring Worldwide Culinary Habits on the Web
• Convergence Analysis of Optimization Algorithms
• New methods to assess and improve LIGO detector duty cycle
• Classifying the formation processes of S0 galaxies using Convolutional Neural Networks
• Selecting Relevant Features from a Multi-domain Representation for Few-shot Classification
• Active Discriminative Text Representation Learning
• TRIE: End-to-End Text Reading and Information Extraction for Document Understanding
• A GRU-based Encoder-Decoder Approach with Attention for Online Handwritten Mathematical Expression Recognition
• RRA: Recurrent Residual Attention for Sequence Learning
• A Multi-Object Rectified Attention Network for Scene Text Recognition
• An Attention-Based Word-Level Interaction Model: Relation Detection for Knowledge Base Question Answering
• Survey2Survey: A deep learning generative model approach for cross-survey image mapping
• DenseRAN for Offline Handwritten Chinese Character Recognition
• TensorDiffEq: Scalable Multi-GPU Forward and Inverse Solvers for Physics Informed Neural Networks
• An improved hybrid CTC-Attention model for speech recognition
• Neural Stain Normalization and Unsupervised Classification of Cell Nuclei in Histopathological Breast Cancer Images
• Sentiment Adaptive End-to-End Dialog Systems
• Constrained Deep Learning using Conditional Gradient and Applications in Computer Vision
• Hyperbolic Deep Neural Networks: A Survey
• Neural networks in quantum many-body physics: a hands-on tutorial
• Vector representations of text data in deep learning
• Towards Exploiting Background Knowledge for Building Conversation Systems
• InverseFaceNet: Deep Monocular Inverse Face Rendering
• The Garden of Forking Paths: Towards Multi-Future Trajectory Prediction
• A limited-size ensemble of homogeneous CNN/LSTMs for high-performance word classification
• Gated Embeddings in End-to-End Speech Recognition for Conversational-Context Fusion
• Learning RGB-D Salient Object Detection using background enclosure, depth contrast, and top-down features
• Creative Robot Dance with Variational Encoder
• T2LR-Net: An unrolling network learning transformed tensor low-rank prior for dynamic MR image reconstruction
• Neural Machine Translation with Word Predictions
• Translating Phrases in Neural Machine Translation
• Deep Transfer Learning for Classification of Variable Sources
• Integrating both Visual and Audio Cues for Enhanced Video Caption
• Exploiting long-term temporal dynamics for video captioning
• A Physical Model for Microstructural Characterization and Segmentation of 3D Tomography Data
• A Deep Bag-of-Features Model for Music Auto-Tagging
• The percolating cluster is invisible to image recognition with deep learning
• Post-breach Recovery: Protection against White-box Adversarial Examples for Leaked DNN Models
• What made you do this? Understanding black-box decisions with sufficient input subsets
• DBTagger: Multi-Task Learning for Keyword Mapping in NLIDBs Using Bi-Directional Recurrent Neural Networks
• An adaptive surrogate modeling based on deep neural networks for large-scale Bayesian inverse problems
• Speaker identification from the sound of the human breath
• Transfer Learning for Human Activity Recognition using Representational Analysis of Neural Networks
• Stochastic Sign Descent Methods: New Algorithms and Better Theory
• Joint Online Parameter Estimation and Optimal Sensor Placement for the Partially Observed Stochastic Advection-Diffusion Equation
• SurvivalNet: Predicting patient survival from diffusion weighted magnetic resonance images using cascaded fully convolutional and 3D convolutional neural networks
• A structural optimization algorithm with stochastic forces and stresses
• Deep Q-Networks for Accelerating the Training of Deep Neural Networks
• Unconstrained Scene Text and Video Text Recognition for Arabic Script
• Autonomous Deep Learning: A Genetic DCNN Designer for Image Classification
• Convolution-Weight-Distribution Assumption: Rethinking the Criteria of Channel Pruning
• Deep Technology Tracing for High-tech Companies
• Using Sentence Plausibility to Learn the Semantics of Transitive Verbs
• Empirical modeling and hybrid machine learning framework for nucleate pool boiling on microchannel structured surfaces
• Compositional ADAM: An Adaptive Compositional Solver
• Pontryagin Differentiable Programming: An End-to-End Learning and Control Framework
• AdaS: Adaptive Scheduling of Stochastic Gradients
• Controlling the Remixing of Separated Dialogue with a Non-Intrusive Quality Estimate
• Multi-layer Representation Learning for Medical Concepts
• Accelerated Large Batch Optimization of BERT Pretraining in 54 minutes
• Stochastic Gradient Descent: Going As Fast As Possible But Not Faster
• Span-Based Constituency Parsing with a Structure-Label System and Provably Optimal Dynamic Oracles
• Online Nonlinear Estimation via Iterative L2-Space Projections: Reproducing Kernel of Subspace
• Layer-wise Learning of Stochastic Neural Networks with Information Bottleneck
• Risk management with machine-learning-based algorithms
• Deep learning for time series classification
• DIMENSION: Dynamic MR Imaging with Both K-space and Spatial Prior Knowledge Obtained via Multi-Supervised Network Training
• A CADe System for Gliomas in Brain MRI using Convolutional Neural Networks
• Generalize Symbolic Knowledge With Neural Rule Engine
• Multi-Hop Paragraph Retrieval for Open-Domain Question Answering
• Sequence-to-Nuggets: Nested Entity Mention Detection via Anchor-Region Networks
• An Empirical Exploration of Skip Connections for Sequential Tagging
• Distilling BERT into Simple Neural Networks with Unlabeled Transfer Data
• Large-Scale Deep Learning Optimizations: A Comprehensive Survey
• Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling
• IMS-Speech: A Speech to Text Tool
• Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction
• An Adaptive Gradient Method with Energy and Momentum
• Nonparametric Neural Networks
• Instantiation
• Rethinking Irregular Scene Text Recognition
• Recurrent Calibration Network for Irregular Text Recognition
• Operations Guided Neural Networks for High Fidelity Data-To-Text Generation
• NDT: Neual Decision Tree Towards Fully Functioned Neural Graph
• Distributed Deep Learning for Question Answering
• Sarcasm SIGN: Interpreting Sarcasm with Sentiment Based Monolingual Machine Translation
• Stochastic Gradient Descent with Nonlinear Conjugate Gradient-Style Adaptive Momentum
• Streaming Word Embeddings with the Space-Saving Algorithm
• Kalman meets Bellman: Improving Policy Evaluation through Value Tracking
• Neural Machine Translation Advised by Statistical Machine Translation
• End-to-end Speech Recognition with Word-based RNN Language Models
• Neural Network Coding
• Recurrent Neural Network based Part-of-Speech Tagger for Code-Mixed Social Media Text
• Neural Density Estimation and Likelihood-free Inference
• Multimodal Memory Modelling for Video Captioning
• Trusting SVM for Piecewise Linear CNNs
• Generative Knowledge Transfer for Neural Language Models
• Toward a full-scale neural machine translation in production: the Booking.com use case
• Neural Networks Regularization Through Class-wise Invariant Representation Learning
• TrackNet: A Deep Learning Network for Tracking High-speed and Tiny Objects in Sports Applications
• A Curriculum Domain Adaptation Approach to the Semantic Segmentation of Urban Scenes
• TAdam: A Robust Stochastic Gradient Optimizer
• Disentangling Adaptive Gradient Methods from Learning Rates
• Decoupled Attention Network for Text Recognition
• Graph-based Neural Sentence Ordering
• Bidirectional Scene Text Recognition with a Single Decoder
• Parton distribution functions
• Adaptive First-and Zeroth-order Methods for Weakly Convex Stochastic Optimization Problems
• Does Yoga Make You Happy? Analyzing Twitter User Happiness using Textual and Temporal Information
• Now You See It, Now You Dont: Adversarial Vulnerabilities in Computational Pathology
• Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformer
• Multi-Label Logo Recognition and Retrieval based on Weighted Fusion of Neural Features
• Improving Transferability of Domain Adaptation Networks Through Domain Alignment Layers
• Spatial Graph Attention and Curiosity-driven Policy for Antiviral Drug Discovery
• Fruit classification using deep feature maps in the presence of deceptive similar classes
• Is SGD a Bayesian sampler? Well, almost
• Editable Neural Networks
• Learn to Augment: Joint Data Augmentation and Network Optimization for Text Recognition
• Adaptive Temporal Difference Learning with Linear Function Approximation
• Empirical study towards understanding line search approximations for training neural networks
• MDE: Multiple Distance Embeddings for Link Prediction in Knowledge Graphs
• Implicit Regularization of Stochastic Gradient Descent in Natural Language Processing: Observations and Implications
• An attention-based Bi-GRU-CapsNet model for hypernymy detection between compound entities
• Semantic Segmentation of Human Thigh Quadriceps Muscle in Magnetic Resonance Images
• GANerated Hands for Real-time 3D Hand Tracking from Monocular RGB
• Visually Grounded Word Embeddings and Richer Visual Features for Improving Multimodal Neural Machine Translation
• Incremental Parsing with Minimal Features Using Bi-Directional LSTM
• Vocabulary Manipulation for Neural Machine Translation
• Distilling Model Knowledge
• Uncertainty-Aware Lookahead Factor Models for Quantitative Investing
• An end-to-end convolutional selective autoencoder approach to Soybean Cyst Nematode eggs detection
• Constraining the three-dimensional orbits of galaxies under ram pressure stripping with convolutional neural networks
• Label Denoising Adversarial Network (LDAN) for Inverse Lighting of Face Images
• Maximum Entropy Flow Networks
• Patient trajectory prediction in the Mimic-III dataset, challenges and pitfalls
• Stack operation of tensor networks
• Deep Quantization: Encoding Convolutional Activations with Deep Generative Model
• Textually Enriched Neural Module Networks for Visual Question Answering
• Genetic Programming and Gradient Descent: A Memetic Approach to Binary Image Classification
• A Decentralized Adaptive Momentum Method for Solving a Class of Min-Max Optimization Problems
• Stochastic Gradient Descent with Polyak's Learning Rate
• Aggregated Momentum: Stability Through Passive Damping
• Accelerating Minibatch Stochastic Gradient Descent using Typicality Sampling
• Smart Library: Identifying Books in a Library using Richly Supervised Deep Scene Text Reading
• Multi-dimensional Gated Recurrent Units for Automated Anatomical Landmark Localization
• Feature Assisted bi-directional LSTM Model for Protein-Protein Interaction Identification from Biomedical Texts
• Character-Level Language Modeling with Hierarchical Recurrent Neural Networks
• Training Deep Networks without Learning Rates Through Coin Betting
• Learning Whole-Image Descriptors for Real-time Loop Detection andKidnap Recovery under Large Viewpoint Difference
• U-Net Based Architecture for an Improved Multiresolution Segmentation in Medical Images
• A Systematic Approach to Featurization for Cancer Drug Sensitivity Predictions with Deep Learning
• The Bayesian Learning Rule
• RSL19BD at DBDC4: Ensemble of Decision Tree-based and LSTM-based Models
• Word and Document Embeddings based on Neural Network Approaches
• Facial Landmarks Detection by Self-Iterative Regression based Landmarks-Attention Network
• Implementation of Stochastic Quasi-Newton's Method in PyTorch
• The Wang-Landau Algorithm as Stochastic Optimization and Its Acceleration
• A machine learning approach to drug repositioning based on drug expression profiles: Applications to schizophrenia and depression/anxiety disorders
• A survey on Bayesian inference for Gaussian mixture model
• VIP -- Variational Inversion Package with example implementations of Bayesian tomographic imaging
• Scalable Gaussian Process Classification via Expectation Propagation
• Meta-Learning Multi-task Communication
• Exemplar Auditing for Multi-Label Biomedical Text Classification
• Dangers of Bayesian Model Averaging under Covariate Shift
• Adaloss: Adaptive Loss Function for Landmark Localization
• Neural SDEs as Infinite-Dimensional GANs
• Wikipedia Vandal Early Detection: from User Behavior to User Embedding
• Neither Quick Nor Proper -- Evaluation of QuickProp for Learning Deep Neural Networks
• Training Neural Networks for and by Interpolation
• Towards Boosting the Accuracy of Non-Latin Scene Text Recognition
• Representational Capacity of Deep Neural Networks -- A Computing Study
• Cross-Modal Conceptualization in Bottleneck Models
• Stroke Constrained Attention Network for Online Handwritten Mathematical Expression Recognition
• U-CNNpred: A Universal CNN-based Predictor for Stock Markets
• Neural Response Generation with Meta-Words
• Improved End-to-End Dysarthric Speech Recognition via Meta-learning Based Model Re-initialization
• Trajectory-based Radical Analysis Network for Online Handwritten Chinese Character Recognition
• End-to-End ASR for Code-switched Hindi-English Speech
• A Neural Spiking Approach Compared to Deep Feedforward Networks on Stepwise Pixel Erasement
• Deeply Learned Preselection of Higgs Dijet Decays at Future Lepton Colliders
• Automatically Generating Commit Messages from Diffs using Neural Machine Translation
• Lightweight and Unobtrusive Data Obfuscation at IoT Edge for Remote Inference
• On Accelerating Distributed Convex Optimizations
• What If We Only Use Real Datasets for Scene Text Recognition? Toward Scene Text Recognition With Fewer Labels
• Sentiment Analysis Using Averaged Weighted Word Vector Features
• Variational Autoencoder with Implicit Optimal Priors
• Coherence-Aware Neural Topic Modeling
• Very Deep Multilingual Convolutional Neural Networks for LVCSR
• On Event Causality Detection in Tweets
• Independent language modeling architecture for end-to-end ASR
• Greedy Search with Probabilistic N-gram Matching for Neural Machine Translation
• Feature based Sequential Classifier with Attention Mechanism
• LSTM knowledge transfer for HRV-based sleep staging
• HAS-QA: Hierarchical Answer Spans Model for Open-domain Question Answering
• Refining Source Representations with Relation Networks for Neural Machine Translation
• Deep Low-Density Separation for Semi-Supervised Classification
• Dissecting Contextual Word Embeddings: Architecture and Representation
• Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding
• BPE and CharCNNs for Translation of Morphology: A Cross-Lingual Comparison and Analysis
• Horizontally Fused Training Array: An Effective Hardware Utilization Squeezer for Training Novel Deep Learning Models
• Transfer learning of language-independent end-to-end ASR with language model fusion
• Convolutional Geometric Matrix Completion
• Leveraging Financial News for Stock Trend Prediction with Attention-Based Recurrent Neural Network
• Multi-task Learning over Graph Structures
• Evolutionary Multi-objective Optimisation in Neurotrajectory Prediction
• Counterfactual Learning from Human Proofreading Feedback for Semantic Parsing
• One for All: Neural Joint Modeling of Entities and Events
• Morphological Inflection Generation with Hard Monotonic Attention
• Sentence Modeling via Multiple Word Embeddings and Multi-level Comparison for Semantic Textual Similarity
• DeepOBS: A Deep Learning Optimizer Benchmark Suite
• RecSys-DAN: Discriminative Adversarial Networks for Cross-Domain Recommender Systems
• Robust and efficient algorithms for high-dimensional black-box quantum optimization
• Dependency Sensitive Convolutional Neural Networks for Modeling Sentences and Documents
• Asymptotic behaviour of learning rates in Armijo's condition
• Projection-Free Adaptive Gradients for Large-Scale Optimization
• Extended pipeline for content-based feature engineering in music genre recognition
• Tensorized Self-Attention: Efficiently Modeling Pairwise and Global Dependencies Together
• Learning Semantically Coherent and Reusable Kernels in Convolution Neural Nets for Sentence Classification
• MLtuner: System Support for Automatic Machine Learning Tuning
• GPU Accelerated Sub-Sampled Newton's Method
• DeepRemaster: Temporal Source-Reference Attention Networks for Comprehensive Video Enhancement
• Multi-Modal Data Augmentation for End-to-End ASR
• Directly Estimating the Variance of the λ-Return Using Temporal-Difference Methods
• A New Deep Learning Method for Image Deblurring in Optical Microscopic Systems
• Trainable Activation Function in Image Classification
• Automatic Post-Editing for Machine Translation
• Gradient descent revisited via an adaptive online learning rate
• Mining Fix Patterns for FindBugs Violations
• Multilingual End-to-End Speech Translation
• AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Fly
• An Asymptotically Optimal Multi-Armed Bandit Algorithm and Hyperparameter Optimization
• Online Sequence Training of Recurrent Neural Networks with Connectionist Temporal Classification
• MAVEN: A Massive General Domain Event Detection Dataset
• REX: Revisiting Budgeted Training with an Improved Schedule
• MobInsight: A Framework Using Semantic Neighborhood Features for Localized Interpretations of Urban Mobility
• On the Convergence of Stochastic Gradient Descent with Bandwidth-based Step Size
• AI based Algorithms of Path Planning, Navigation and Control for Mobile Ground Robots and UAVs
• TI-POOLING: transformation-invariant pooling for feature learning in Convolutional Neural Networks
• M${}^2$S-Net: Multi-Modal Similarity Metric Learning based Deep Convolutional Network for Answer Selection
• DS-MLR: Exploiting Double Separability for Scaling up Distributed Multinomial Logistic Regression
• Reconstruct and Represent Video Contents for Captioning via Reinforcement Learning
• EdgeNet: A novel approach for Arabic numeral classification
• Theme-aware generation model for chinese lyrics
• Estimating initial conditions for dynamical systems with incomplete information
• Can Machine Generate Traditional Chinese Poetry? A Feigenbaum Test
• Interpretable and synergistic deep learning for visual explanation and statistical estimations of segmentation of disease features from medical images
• An Effective Label Noise Model for DNN Text Classification
• Aggregating explanation methods for stable and robust explainability
• LOSSGRAD: automatic learning rate in gradient descent
• Topology Optimization under Uncertainty using a Stochastic Gradient-based Approach
• Block-Randomized Stochastic Proximal Gradient for Low-Rank Tensor Factorization
• Lattice CNNs for Matching Based Chinese Question Answering
• Causal Attention for Vision-Language Tasks
• Deep Multi-camera People Detection
• FPGA-Based Low-Power Speech Recognition with Recurrent Neural Networks
• Exploiting Adam-like Optimization Algorithms to Improve the Performance of Convolutional Neural Networks
• Emulating quantum computation with artificial neural networks
• Trust Region Value Optimization using Kalman Filtering
• Primitive Representation Learning for Scene Text Recognition
• Light-Weighted CNN for Text Classification
• Neural Networks with Structural Resistance to Adversarial Attacks
• Dark Matter Subhalos, Strong Lensing and Machine Learning
• Deep Image Orientation Angle Detection
• A simulation study to distinguish prompt photon from ${\pi }^0$ and beam halo in a granular calorimeter using deep networks
• Gradient descent with momentum --- to accelerate or to super-accelerate?
• An Efficient Character-Level Neural Machine Translation
• Newton-ADMM: A Distributed GPU-Accelerated Optimizer for Multiclass Classification Problems
• $n$-ML: Mitigating Adversarial Examples via Ensembles of Topologically Manipulated Classifiers
• Real-Time Automated Answer Scoring
• A Feature Learning Siamese Model for Intelligent Control of the Dynamic Range Compressor
• Stochastic Anderson Mixing for Nonconvex Stochastic Optimization
• Read, Tag, and Parse All at Once, or Fully-neural Dependency Parsing
• Stochastic Gradient Descent with Hyperbolic-Tangent Decay on Classification
• Fixed-Point Performance Analysis of Recurrent Neural Networks
• Nugget Proposal Networks for Chinese Event Detection
• AutoSTR: Efficient Backbone Search for Scene Text Recognition
• Tune smarter not harder: A principled approach to tuning learning rates for shallow nets
• Neural Mixture Distributional Regression
• Moment-Based Domain Adaptation: Learning Bounds and Algorithms
• New Insight in Cervical Cancer Diagnosis Using Convolution Neural Network Architecture
• Refining Source Representations with Relation Networks for Neural Machine Translation
• Exploring Machine Speech Chain for Domain Adaptation and Few-Shot Speaker Adaptation
• Fusing Recency into Neural Machine Translation with an Inter-Sentence Gate Model
• Semi-Supervised Learning Enabled by Multiscale Deep Neural Network Inversion
• A Simple Dynamic Learning Rate Tuning Algorithm For Automated Training of DNNs
• Monocular Real-Time Volumetric Performance Capture
• JotlasNet: Joint Tensor Low-Rank and Attention-based Sparse Unrolling Network for Accelerating Dynamic MRI
• Large Vocabulary Automatic Chord Estimation Using Deep Neural Nets: Design Framework, System Variations and Limitations
• A Dynamical View on Optimization Algorithms of Overparameterized Neural Networks
• Riemannian Adaptive Optimization Algorithm and Its Application to Natural Language Processing
• Orthogonal Over-Parameterized Training
• $\mathcal{G}$-SGD: Optimizing ReLU Neural Networks in its Positively Scale-Invariant Space
• Cross-Attention End-to-End ASR for Two-Party Conversations
• Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution Data
• Unconstrained optimisation on Riemannian manifolds
• Learning Acoustic Word Embeddings with Temporal Context for Query-by-Example Speech Search
• Unsupervised Transfer of Semantic Role Models from Verbal to Nominal Domain
• A Variational Approach to Weakly Supervised Document-Level Multi-Aspect Sentiment Classification
• Quad-networks: unsupervised learning to rank for interest point detection
• Learning and Visualizing Localized Geometric Features Using 3D-CNN: An Application to Manufacturability Analysis of Drilled Holes
• Auto-Encoder based Co-Training Multi-View Representation Learning
• Radical analysis network for zero-shot learning in printed Chinese character recognition
• APMSqueeze: A Communication Efficient Adam-Preconditioned Momentum SGD Algorithm
• Learning to Compose Topic-Aware Mixture of Experts for Zero-Shot Video Captioning
• Mixture of Experts Distributional Regression: Implementation Using Robust Estimation with Adaptive First-order Methods
• MultiNet: Multi-Modal Multi-Task Learning for Autonomous Driving
• Learning to Discriminate Noises for Incorporating External Information in Neural Machine Translation
• Learned-Norm Pooling for Deep Feedforward and Recurrent Neural Networks
• Chinese-Portuguese Machine Translation: A Study on Building Parallel Corpora from Comparable Texts
• The Nonlinearity Coefficient - A Practical Guide to Neural Architecture Design
• Contextual and Position-Aware Factorization Machines for Sentiment Classification
• Dimension-free convergence rates for gradient Langevin dynamics in RKHS
• Strange Jet Tagging
• Hot Swapping for Online Adaptation of Optimization Hyperparameters
• Autoencoder Regularized Network For Driving Style Representation Learning
• Learning From Graph Neighborhoods Using LSTMs
• Sentiment Analysis of German Twitter
• Cooperative Learning of Disjoint Syntax and Semantics
• Multi Task Deep Morphological Analyzer: Context Aware Joint Morphological Tagging and Lemma Prediction
• TRk-CNN: Transferable Ranking-CNN for image classification of glaucoma, glaucoma suspect, and normal eyes
• Efficient Elastic Net Regularization for Sparse Linear Models
• Traditional Chinese Synthetic Datasets Verified with Labeled Data for Scene Text Recognition
• Neural Simile Recognition with Cyclic Multitask Learning and Local Attention
• Scale-Consistent Fusion: from Heterogeneous Local Sampling to Global Immersive Rendering
• Variational inference for cutting feedback in misspecified models
• Modeling and Predicting Popularity Dynamics via Deep Learning Attention Mechanism
• Effective Sentence Scoring Method using Bidirectional Language Model for Speech Recognition
• Bi-fidelity Stochastic Gradient Descent for Structural Optimization under Uncertainty
• Citadel: Protecting Data Privacy and Model Confidentiality for Collaborative Learning with SGX
• Incorporating Domain Knowledge To Improve Topic Segmentation Of Long MOOC Lecture Videos
• SRR-Net: A Super-Resolution-Involved Reconstruction Method for High Resolution MR Imaging
• Topic Identification for Speech without ASR
• Differentiable Visual Computing
• Salience Estimation with Multi-Attention Learning for Abstractive Text Summarization
• Improving Long-Tail Relation Extraction with Collaborating Relation-Augmented Attention
• Diacritization of Maghrebi Arabic Sub-Dialects
• A new way to constrain the densities of intra-group medium in groups of galaxies with convolutional neural networks
• Evolving Neural Networks in Reinforcement Learning by means of UMDAc
• Stage-based Hyper-parameter Optimization for Deep Learning
• Modularity Matters: Learning Invariant Relational Reasoning Tasks
• An Exploratory Analysis of the Latent Structure of Process Data via Action Sequence Autoencoder
• On the asymptotic rate of convergence of Stochastic Newton algorithms and their Weighted Averaged versions
• End-to-end NILM System Using High Frequency Data and Neural Networks
• Attention Focusing for Neural Machine Translation by Bridging Source and Target Embeddings
• Applying Data Augmentation to Handwritten Arabic Numeral Recognition Using Deep Learning Neural Networks
• Transfer learning from language models to image caption generators: Better models may not transfer better
• AON: Towards Arbitrarily-Oriented Text Recognition
• Adaptive Physics-Informed Neural Networks for Markov-Chain Monte Carlo
• Leveraging Class Similarity to Improve Deep Neural Network Robustness
• Dynamic Sentence Sampling for Efficient Training of Neural Machine Translation
• A New Perspective for Flexible Feature Gathering in Scene Text Recognition Via Character Anchor Pooling
• Expectigrad: Fast Stochastic Optimization with Robust Convergence Properties
• A Deep Neural Architecture for Sentence-level Sentiment Classification in Twitter Social Networking
• The AMU-UEDIN Submission to the WMT16 News Translation Task: Attention-based NMT Models as Feature Functions in Phrase-based SMT
• Deep Decomposition Network for Image Processing: A Case Study for Visible and Infrared Image Fusion
• Bayesian Hidden Physics Models: Uncertainty Quantification for Discovery of Nonlinear Partial Differential Operators from Data
• Deep Low-rank plus Sparse Network for Dynamic MR Imaging
• Learning to be Global Optimizer
• Improving the convergence of SGD through adaptive batch sizes
• Understanding Dropout: Training Multi-Layer Perceptrons with Auxiliary Independent Stochastic Neurons
• Joint Learning of Intrinsic Images and Semantic Segmentation
• 1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speed
• Near-Data Processing for Differentiable Machine Learning Models
• Event-Based Modeling with High-Dimensional Imaging Biomarkers for Estimating Spatial Progression of Dementia
• Attention-based Neural Text Segmentation
• SPIN: Structure-Preserving Inner Offset Network for Scene Text Recognition
• Neumann Optimizer: A Practical Optimization Algorithm for Deep Neural Networks
• Novel Suboptimal approaches for Hyperparameter Tuning of Deep Neural Network [under the shelf of Optical Communication]
• Deep convolutional acoustic word embeddings using word-pair side information
• Neuro-Optimization: Learning Objective Functions Using Neural Networks
• Text Length Adaptation in Sentiment Classification
• Improving Online Forums Summarization via Hierarchical Unified Deep Neural Network
• ECC: Platform-Independent Energy-Constrained Deep Neural Network Compression via a Bilinear Regression Model
• Multiple Range-Restricted Bidirectional Gated Recurrent Units with Attention for Relation Classification
• SubCharacter Chinese-English Neural Machine Translation with Wubi encoding
• SGDLibrary: A MATLAB library for stochastic gradient descent algorithms
• Stochastic optimization for numerical evaluation of imprecise probabilities
• Neural-based Noise Filtering from Word Embeddings
• FedOCR: Communication-Efficient Federated Learning for Scene Text Recognition
• Stochastic Gradient Langevin Dynamics Algorithms with Adaptive Drifts
• Deep Learning Service for Efficient Data Distribution Aware Sorting
• Generating Thematic Chinese Poetry using Conditional Variational Autoencoders with Hybrid Decoders
• Stabilizing Inputs to Approximated Nonlinear Functions for Inference with Homomorphic Encryption in Deep Neural Networks
• Dependency Parsing with LSTMs: An Empirical Evaluation
• Faster Stochastic Variational Inference using Proximal-Gradient Methods with General Divergence Functions
• On the One-sided Convergence of Adam-type Algorithms in Non-convex Non-concave Min-max Optimization
• Scalable Neural Network Compression and Pruning Using Hard Clustering and L1 Regularization
• Similarity-based Text Recognition by Deeply Supervised Siamese Network
• Tree-Structured Neural Machine for Linguistics-Aware Sentence Generation
• Efficient Designs of SLOPE Penalty Sequences in Finite Dimension
• Data Augmentation with Locally-time Reversed Speech for Automatic Speech Recognition
• Learning Enabled Dense Space-division Multiplexing through a Single Multimode Fibre
• Neural Architectures for Named Entity Recognition
• Efficient and Accurate Gradients for Neural SDEs
• Unsupervised preprocessing for Tactile Data
• S3Pool: Pooling with Stochastic Spatial Sampling
• Optimal Complexity in Decentralized Training
• Deep Unsupervised Multi-View Detection of Video Game Stream Highlights
• Learn Faster and Forget Slower via Fast and Stable Task Adaptation
• Drill the Cork of Information Bottleneck by Inputting the Most Important Data
• Learning Word Sense Embeddings from Word Sense Definitions
• Stochastic Gradient Methods with Block Diagonal Matrix Adaptation
• Recent Advances in Convolutional Neural Network Acceleration
• Siamese convolutional networks based on phonetic features for cognate identification
• Inductive Visual Localisation: Factorised Training for Superior Generalisation
• Hippo: Taming Hyper-parameter Optimization of Deep Learning with Stage Trees
• Analysis of CNN-based remote-PPG to understand limitations and sensitivities
• SAdam: A Variant of Adam for Strongly Convex Functions
• Recurrent Point Review Models
• Multiple Sclerosis Lesion Segmentation -- A Survey of Supervised CNN-Based Methods
• Replicated Siamese LSTM in Ticketing System for Similarity Learning and Retrieval in Asymmetric Texts
• Gravity Optimizer: a Kinematic Approach on Optimization in Deep Learning
• ConvoSource: Radio-Astronomical Source-Finding with Convolutional Neural Networks
• An Optimistic Acceleration of AMSGrad for Nonconvex Optimization
• Learning the Step-size Policy for the Limited-Memory Broyden-Fletcher-Goldfarb-Shanno Algorithm
• Design of Capacity-Approaching Low-Density Parity-Check Codes using Recurrent Neural Networks
• From Recognition to Prediction: Analysis of Human Action and Trajectory Prediction in Video
• Incorporating the Barzilai-Borwein Adaptive Step Size into Sugradient Methods for Deep Network Training
• Gradient Boosted Decision Tree Neural Network
• An End-to-End Neural Network for Polyphonic Piano Music Transcription
• Time-Delay Momentum: A Regularization Perspective on the Convergence and Generalization of Stochastic Momentum for Deep Learning
• Deep Representation for Connected Health: Semi-supervised Learning for Analysing the Risk of Urinary Tract Infections in People with Dementia
• Adam with Bandit Sampling for Deep Learning
• PIE: Portrait Image Embedding for Semantic Control
• Improving Distant Supervised Relation Extraction by Dynamic Neural Network
• How Do Source-side Monolingual Word Embeddings Impact Neural Machine Translation?
• Promising Accurate Prefix Boosting for sequence-to-sequence ASR
• Representation Learning of Entities and Documents from Knowledge Base Descriptions
• Language model integration based on memory control for sequence to sequence speech recognition
• CNN-based MultiChannel End-to-End Speech Recognition for everyday home environments
• Improving Optimization for Models With Continuous Symmetry Breaking
• Hybrid Dialog State Tracker with ASR Features
• Optimizer Fusion: Efficient Training with Better Locality and Parallelism
• Adversarial Heart Attack: Neural Networks Fooled to Segment Heart Symbols in Chest X-Ray Images
• Local Distance Metric Learning for Nearest Neighbor Algorithm
• A Corpus for English-Japanese Multimodal Neural Machine Translation with Comparable Sentences
• AnoNet: Weakly Supervised Anomaly Detection in Textured Surfaces
• Parallel Scale-wise Attention Network for Effective Scene Text Recognition
• Toward Model Parallelism for Deep Neural Network based on Gradient-free ADMM Framework
• Learn distributed GAN with Temporary Discriminators
• Towards Deep Compositional Networks
• Joint Multitask Learning for Community Question Answering Using Task-Specific Embeddings
• Audio Based Disambiguation Of Music Genre Tags
• Full Workspace Generation of Serial-link Manipulators by Deep Learning based Jacobian Estimation
• A Variational Approach to Unsupervised Sentiment Analysis
• A Survey on Large-scale Machine Learning
• Evaluating computational models of infant phonetic learning across languages
• Deep Multi-view Models for Glitch Classification
• "I have vxxx bxx connexxxn!": Facing Packet Loss in Deep Speech Emotion Recognition
• Ensemble Transfer Learning for Emergency Landing Field Identification on Moderate Resource Heterogeneous Kubernetes Cluster
• Exploiting Effective Representations for Chinese Sentiment Analysis Using a Multi-Channel Convolutional Neural Network
• Dialog-context aware end-to-end speech recognition
• PatchBatch: a Batch Augmented Loss for Optical Flow
• Decoupling Visual-Semantic Feature Learning for Robust Scene Text Recognition
• Oracle Teacher: Leveraging Target Information for Better Knowledge Distillation of CTC Models
• TIME: A Transparent, Interpretable, Model-Adaptive and Explainable Neural Network for Dynamic Physical Processes
• Learning User Intent from Action Sequences on Interactive Systems
• Deep Low-rank Prior in Dynamic MR Imaging
• Remedies against the Vocabulary Gap in Information Retrieval
• Multiple Instance Learning Networks for Fine-Grained Sentiment Analysis
• Specular reflections removal in colposcopic images based on neural networks: Supervised training with no ground truth previous knowledge
• Keyword-based Query Comprehending via Multiple Optimized-Demand Augmentation
• Layer-wise Conditioning Analysis in Exploring the Learning Dynamics of DNNs
• SA-GD: Improved Gradient Descent Learning Strategy with Simulated Annealing
• On the Trend-corrected Variant of Adaptive Stochastic Optimization Methods
• ADINE: An Adaptive Momentum Method for Stochastic Gradient Descent
• Neural Machine Translation: A Review and Survey
• Tag, Copy or Predict: A Unified Weakly-Supervised Learning Framework for Visual Information Extraction using Sequences
• Initializing ReLU networks in an expressive subspace of weights
• Sequential Diagnosis Prediction with Transformer and Ontological Representation
• Formal models of Structure Building in Music, Language and Animal Songs
• Momentum Centering and Asynchronous Update for Adaptive Gradient Methods
• Universal Online Convex Optimization Meets Second-order Bounds
• Self-Attention-Based Message-Relevant Response Generation for Neural Conversation Model
• LLM4Laser: Large Language Models Automate the Design of Lasers
• Acoustics-specific Piano Velocity Estimation
• A proof of concept study for machine learning application to stenosis detection
• Optimizations of Autoencoders for Analysis and Classification of Microscopic In Situ Hybridization Images
• On Adaptive Stochastic Optimization for Streaming Data: A Newton's Method with O(dN) Operations
• Stochastic Gradient Variance Reduction by Solving a Filtering Problem
• Deep inference of simulated strong lenses in ground-based surveys
• End-to-end analysis using image classification
• AutoDrop: Training Deep Learning Models with Automatic Learning Rate Drop
• Deep Structure Learning using Feature Extraction in Trained Projection Space
• Hyperspectral Band Selection for Multispectral Image Classification with Convolutional Networks
• Memory Augmented Optimizers for Deep Learning
• AdaL: Adaptive Gradient Transformation Contributes to Convergences and Generalizations
• On Hyper-parameter Tuning for Stochastic Optimization Algorithms
• L2M: Practical posterior Laplace approximation with optimization-driven second moment estimation
• Distinguishing Antonyms and Synonyms in a Pattern-based Neural Network
• Dual Stochastic Natural Gradient Descent and convergence of interior half-space gradient approximations
• Questions to Guide the Future of Artificial Intelligence Research
• LassoLayer: Nonlinear Feature Selection by Switching One-to-one Links
• Beyond Textual Data: Predicting Drug-Drug Interactions from Molecular Structure Images using Siamese Neural Networks
• Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Text
• Neural Multi-Source Morphological Reinflection
• Learning review representations from user and product level information for spam detection
• Boltzmann machine learning and regularization methods for inferring evolutionary fields and couplings from a multiple sequence alignment
• Adaptive Embedding Gate for Attention-Based Scene Text Recognition
• A Distributed Training Algorithm of Generative Adversarial Networks with Quantized Gradients
• AttentionBoost: Learning What to Attend by Boosting Fully Convolutional Networks
• From Note-Level to Chord-Level Neural Network Models for Voice Separation in Symbolic Music
• Self-Tuning Stochastic Optimization with Curvature-Aware Gradient Filtering
• Many could be better than all: A novel instance-oriented algorithm for Multi-modal Multi-label problem
• Privileged Features Distillation at Taobao Recommendations
• DEAM: Adaptive Momentum with Discriminative Weight for Stochastic Optimization
• Image-Based Jet Analysis
• On SGD's Failure in Practice: Characterizing and Overcoming Stalling
• CADA: Communication-Adaptive Distributed Adam
• Generalizing Adversarial Examples by AdaBelief Optimizer
• Neuromodulated Learning in Deep Neural Networks
• Learning Representations of Social Media Users
• A Probabilistically Motivated Learning Rate Adaptation for Stochastic Optimization
• Learning to Embed Sentences Using Attentive Recursive Trees
• Deep Manifold Learning for Dynamic MR Imaging
• Exploring the Use of Attention within an Neural Machine Translation Decoder States to Translate Idioms
• POLO: a POLicy-based Optimization library
• ReADS: A Rectified Attentional Double Supervised Network for Scene Text Recognition
• Fast and Accurate Deep Bidirectional Language Representations for Unsupervised Learning
• Data Annealing for Informal Language Understanding Tasks
• Adversarial Learning for Image Forensics Deep Matching with Atrous Convolution
• Textual Analogy Parsing: What's Shared and What's Compared among Analogous Facts
• Predicting the Argumenthood of English Prepositional Phrases
• Sparse data to structured imageset transformation
• ProcData: An R Package for Process Data Analysis
• Studying Attention Models in Sentiment Attitude Extraction Task
• Volumetric performance capture from minimal camera viewpoints
• Deep Autoencoder for Combined Human Pose Estimation and body Model Upscaling
• Trust-Region Algorithms for Training Responses: Machine Learning Methods Using Indefinite Hessian Approximations
• Particle Filtering Methods for Stochastic Optimization with Application to Large-Scale Empirical Risk Minimization
• Optimization of neural networks via finite-value quantum fluctuations
• A New Benchmark and Progress Toward Improved Weakly Supervised Learning
• Gradient-only line searches to automatically determine learning rates for a variety of stochastic training algorithms
• Machine Learning Approach for Transforming Scattering Parameters to Complex Permittivity
• Depth-Adaptive Neural Networks from the Optimal Control viewpoint
• Adma: A Flexible Loss Function for Neural Networks
• Predictive Local Smoothness for Stochastic Gradient Methods
• A Unified Framework for Training Neural Networks
• An Investigation on Deep Learning with Beta Stabilizer
• A Machine-Learning Framework for Design for Manufacturability
• Channel-Directed Gradients for Optimization of Convolutional Neural Networks
• An Analysis of Alternating Direction Method of Multipliers for Feed-forward Neural Networks
• Multi-Head Decoder for End-to-End Speech Recognition
• Integrating Source-channel and Attention-based Sequence-to-sequence Models for Speech Recognition
• On Higher-order Moments in Adam
• A multiobjective deep learning approach for predictive classification in Neuroblastoma
• On Architectures for Including Visual Information in Neural Language Models for Image Description
• Generating an Explainable ECG Beat Space With Variational Auto-Encoders
• Elephant Search with Deep Learning for Microarray Data Analysis
• Progressive VAE Training on Highly Sparse and Imbalanced Data
• Second-order Information in First-order Optimization Methods
• Accumulated Polar Feature-based Deep Learning for Efficient and Lightweight Automatic Modulation Classification with Channel Compensation Mechanism
• Knot Selection in Sparse Gaussian Processes
• Triplet Online Instance Matching Loss for Person Re-identification
• CRDN: Cascaded Residual Dense Networks for Dynamic MR Imaging with Edge-enhanced Loss Constraint
• Debugging Frame Semantic Role Labeling
• SPI-Optimizer: an integral-Separated PI Controller for Stochastic Optimization
• ICLR Reproducibility Challenge Report (Padam : Closing The Generalization Gap Of Adaptive Gradient Methods in Training Deep Neural Networks)
• Prototypical Recurrent Unit
• Optimizing Recurrent Neural Networks Architectures under Time Constraints
• FastReg: Fast Non-Rigid Registration via Accelerated Optimisation on the Manifold of Diffeomorphisms
• Blockwise Adaptivity: Faster Training and Better Generalization in Deep Learning
• A Lightweight Recurrent Network for Sequence Modeling
• A Deep Learning Framework for Classification of in vitro Multi-Electrode Array Recordings
• Power Gradient Descent
• End-to-End Speech Recognition with High-Frame-Rate Features Extraction
• Discourse Marker Augmented Network with Reinforcement Learning for Natural Language Inference
• Recognizing and Curating Photo Albums via Event-Specific Image Importance
• Single-Solution Hypervolume Maximization and its use for Improving Generalization of Neural Networks
• Sequential Matrix Completion
• Image Patch Matching Using Convolutional Descriptors with Euclidean Distance
• BPGrad: Towards Global Optimality in Deep Learning via Branch and Pruning
• Context-modulation of hippocampal dynamics and deep convolutional networks
• Toward Implicit Sample Noise Modeling: Deviation-driven Matrix Factorization
• Communication-Compressed Adaptive Gradient Method for Distributed Nonconvex Optimization
• Predictive coding, precision and natural gradients
• Dynamic Compositionality in Recursive Neural Networks with Structure-aware Tag Representations
• Expressive power of outer product manifolds on feed-forward neural networks
• Efficient Scene Text Detection with Textual Attention Tower
• Text Recognition in Real Scenarios with a Few Labeled Samples
• Leveraging Medical Sentiment to Understand Patients Health on Social Media
• Backtracking gradient descent method for general $C^1$ functions, with applications to Deep Learning
• A deep level set method for image segmentation
• Adaptive Gradient Method with Resilience and Momentum
• Semi-Supervised Event Extraction with Paraphrase Clusters
• Imitation Learning for Neural Morphological String Transduction
• Stochastic Approximate Gradient Descent via the Langevin Algorithm
• Neural Transductive Learning and Beyond: Morphological Generation in the Minimal-Resource Setting
• Representation Learning Models for Entity Search
• Significance of Maximum Spectral Amplitude in Sub-bands for Spectral Envelope Estimation and Its Application to Statistical Parametric Speech Synthesis
• Adam Induces Implicit Weight Sparsity in Rectifier Neural Networks
• Linguistic Typology Features from Text: Inferring the Sparse Features of World Atlas of Language Structures
• Compositional Stochastic Average Gradient for Machine Learning and Related Applications
• Speeding Up Neural Machine Translation Decoding by Cube Pruning
• Learning to Learn Morphological Inflection for Resource-Poor Languages
• When Weak Becomes Strong: Robust Quantification of White Matter Hyperintensities in Brain MRI scans
• Asynchronous Distributed Variational Gaussian Processes for Regression
• SAFE: Scale Aware Feature Encoder for Scene Text Recognition
• Unifying the Stochastic Spectral Descent for Restricted Boltzmann Machines with Bernoulli or Gaussian Inputs
• Cross Copy Network for Dialogue Generation
• Learning Explicit Prosody Models and Deep Speaker Embeddings for Atypical Voice Conversion
• Automatic, Dynamic, and Nearly Optimal Learning Rate Specification by Local Quadratic Approximation
• Multi-Party Dual Learning
• On Anytime Learning at Macroscale
• An acceleration strategy for randomize-then-optimize sampling via deep neural networks
• Predictive Process Model Monitoring using Recurrent Neural Networks
• A New Family of Neural Networks Provably Resistant to Adversarial Attacks
• SAR: Semantic Analysis for Recommendation
• Fault Prediction based on Software Metrics and SonarQube Rules. Machine or Deep Learning?
• Mining Program Properties From Neural Networks Trained on Source Code Embeddings
• Exploiting Domain Knowledge via Grouped Weight Sharing with Application to Text Categorization
• The RLLChatbot: a solution to the ConvAI challenge
• Extreme Tensoring for Low-Memory Preconditioning
• Regularized adversarial examples for model interpretability
• Can Intelligent Hyperparameter Selection Improve Resistance to Adversarial Examples?
• Categorical Metadata Representation for Customized Text Classification
• Learning Representations by Stochastic Meta-Gradient Descent in Neural Networks
• Greedy Step Averaging: A parameter-free stochastic optimization method
• Advance Prediction of Ventricular Tachyarrhythmias using Patient Metadata and Multi-Task Networks
• User-friendly automatic transcription of low-resource languages: Plugging ESPnet into Elpis
• DTN: A Learning Rate Scheme with Convergence Rate of $\mathcal{O}(1/t)$ for SGD
• LSTM based AE-DNN constraint for better late reverb suppression in multi-channel LP formulation
• Local Convergence of Adaptive Gradient Descent Optimizers
• Distilling Interpretable Models into Human-Readable Code
• Train your classifier first: Cascade Neural Networks Training from upper layers to lower layers
• Attribute Acquisition in Ontology based on Representation Learning of Hierarchical Classes and Attributes
• An Adaptive Deep Learning Algorithm Based Autoencoder for Interference Channels
• Inertial Proximal Deep Learning Alternating Minimization for Efficient Neutral Network Training
• Manifold Optimization Assisted Gaussian Variational Approximation
• Parabolic Approximation Line Search for DNNs
• AsymptoticNG: A regularized natural gradient optimization algorithm with look-ahead strategy
• Efficient Semi-Implicit Variational Inference
• A Comprehensive Study on Optimization Strategies for Gradient Descent In Deep Learning
• Neuroevolutionary Feature Representations for Causal Inference
• Data-driven Algorithm Selection and Parameter Tuning: Two Case studies in Optimization and Signal Processing
• An Adaptive Remote Stochastic Gradient Method for Training Neural Networks
• Learning Sentimental Influences from Users' Behaviors
• Collaborative Training of Tensors for Compositional Distributional Semantics
• xRAI: Explainable Representations through AI
• Ada-Segment: Automated Multi-loss Adaptation for Panoptic Segmentation
• Painless step size adaptation for SGD
• Joint Regularization on Activations and Weights for Efficient Neural Network Pruning
• TRACE: Early Detection of Chronic Kidney Disease Onset with Transformer-Enhanced Feature Embedding
• Coarse-to-Fine Memory Matching for Joint Retrieval and Classification
• Atom Responding Machine for Dialog Generation
• Light Propagation Prediction through Multimode Optical Fibers with a Deep Neural Network
• Compositional Sentence Representation from Character within Large Context Text
• Accelerating Distributed SGD for Linear Regression using Iterative Pre-Conditioning
• Acoustic-to-Word Models with Conversational Context Information
• A New Training Method for Feedforward Neural Networks Based on Geometric Contraction Property of Activation Functions
• Adaptive Optimization with Examplewise Gradients
• On Generalization of Adaptive Methods for Over-parameterized Linear Regression
• Towards Pure End-to-End Learning for Recognizing Multiple Text Sequences from an Image
• On tuning deep learning models: a data mining perspective
• AI-lead Court Debate Case Investigation
• A theoretical and empirical study of new adaptive algorithms with additional momentum steps and shifted updates for stochastic non-convex optimization
• Few-Shot Meta-Denoising
• FLAG n' FLARE: Fast Linearly-Coupled Adaptive Gradient Methods
• Improving Open Information Extraction via Iterative Rank-Aware Learning
• Identify the stiffness of DNA via deep learning
• PredProp: Bidirectional Stochastic Optimization with Precision Weighted Predictive Coding
• Neural Universal Discrete Denoiser
• Training Deep Neural Networks via Branch-and-Bound
• One Model to Learn Both: Zero Pronoun Prediction and Translation
• Automatic and Simultaneous Adjustment of Learning Rate and Momentum for Stochastic Gradient Descent
• Incremental Cross-Domain Adaptation for Robust Retinopathy Screening via Bayesian Deep Learning
• Accelerating Parameter Extraction of Power MOSFET Models Using Automatic Differentiation
• Genealogical Population-Based Training for Hyperparameter Optimization
• Towards Quantized Model Parallelism for Graph-Augmented MLPs Based on Gradient-Free ADMM Framework
• On Multilingual Training of Neural Dependency Parsers
• Improving Graph-based Sentence Ordering with Iteratively Predicted Pairwise Orderings
• Uncover the Ground-Truth Relations in Distant Supervision: A Neural Expectation-Maximization Framework
• ${\bar{G}}_{mst}$:An Unbiased Stratified Statistic and a Fast Gradient Optimization Algorithm Based on It
• Attention Based Neural Architecture for Rumor Detection with Author Context Awareness
• Dynamic Regret Analysis for Online Meta-Learning
• Contributions to Large Scale Bayesian Inference and Adversarial Machine Learning
• Toward Communication Efficient Adaptive Gradient Method
• Tom: Leveraging trend of the observed gradients for faster convergence
• COCO Denoiser: Using Co-Coercivity for Variance Reduction in Stochastic Convex Optimization
• Top-Rank Enhanced Listwise Optimization for Statistical Machine Translation
• FastMVAE2: On improving and accelerating the fast variational autoencoder-based source separation algorithm for determined mixtures
• Predictive networking and optimization for flow-based networks
• Ensemble Of Deep Neural Networks For Acoustic Scene Classification
• Multichannel Audio Source Separation with Independent Deeply Learned Matrix Analysis Using Product of Source Models
• Deploying a BERT-based Query-Title Relevance Classifier in a Production System: a View from the Trenches
• Can Edge Probing Tasks Reveal Linguistic Knowledge in QA Models?
• Logit Attenuating Weight Normalization
• Inverse Problem of Nonlinear Schrödinger Equation as Learning of Convolutional Neural Network
• A New Adaptive Gradient Method with Gradient Decomposition
• Lifetime policy reuse and the importance of task capacity
• Data-driven Feature Sampling for Deep Hyperspectral Classification and Segmentation
• A study of an energy-dependent anisotropy of cosmic rays beyond the GZK cut-off with deep neural networks
• Temporal-aware Language Representation Learning From Crowdsourced Labels
• Learning Algebraic Recombination for Compositional Generalization
• Flexible numerical optimization with ensmallen
• Supervised Neural Discrete Universal Denoiser for Adaptive Denoising
• Training for Speech Recognition on Coprocessors
• Block-Cyclic Stochastic Coordinate Descent for Deep Neural Networks
• Convex optimization
• A Corpus for Detecting High-Context Medical Conditions in Intensive Care Patient Notes Focusing on Frequently Readmitted Patients
• Deep3DPose: Realtime Reconstruction of Arbitrarily Posed Human Bodies from Single RGB Images
• Skip-Thought Memory Networks
• Cogradient Descent for Dependable Learning
• Cost-Sensitive Approach to Batch Size Adaptation for Gradient Descent
• Neural Networks Classifier for Data Selection in Statistical Machine Translation
• Convergence Analysis of Gradient Descent Algorithms with Proportional Updates
• Investigating Alternatives to the Root Mean Square for Adaptive Gradient Methods
• Convexification of Neural Graph
• Automation for Interpretable Machine Learning Through a Comparison of Loss Functions to Regularisers
• Double Descent Optimization Pattern and Aliasing: Caveats of Noisy Labels
• A Coarse to Fine Question Answering System based on Reinforcement Learning
• 2nd-order Updates with 1st-order Complexity
• MaxVA: Fast Adaptation of Step Sizes by Maximizing Observed Variance of Gradients
• Performance Analysis of a Foreground Segmentation Neural Network Model
• Manifold Regularized Discriminative Neural Networks
• Deep Cross-Subject Mapping of Neural Activity
• Whitening and second order optimization both make information in the dataset unusable during training, and can reduce or prevent generalization
• Bayesian Recurrent Units and the Forward-Backward Algorithm
• Predicting Discourse Structure using Distant Supervision from Sentiment
• Uncover Sexual Harassment Patterns from Personal Stories by Joint Key Element Extraction and Categorization
• Exploring Lexicon-Free Modeling Units for End-to-End Korean and Korean-English Code-Switching Speech Recognition
• Syntax-Enhanced Self-Attention-Based Semantic Role Labeling
• Single image super-resolution using self-optimizing mask via fractional-order gradient interpolation and reconstruction
• A Deep Q-Learning Agent for the L-Game with Variable Batch Training
• A straightforward line search approach on the expected empirical loss for stochastic deep learning problems
• Understanding the Role of Adversarial Regularization in Supervised Learning
• mL-BFGS: A Momentum-based L-BFGS for Distributed Large-Scale Neural Network Optimization
• MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension
• Neograd: Near-Ideal Gradient Descent
• BAMSProd: A Step towards Generalizing the Adaptive Optimization Methods to Deep Binary Model
• Charged Point Normalization: An Efficient Solution to the Saddle Point Problem
• Face Sketch Synthesis with Style Transfer using Pyramid Column Feature
• Knowledge Graphs for Multilingual Language Translation and Generation
• A multi-view approach for Mandarin non-native mispronunciation verification
• Volatility Forecasting with 1-dimensional CNNs via transfer learning
• Evaluating the relative contribution of data sources in a Bayesian analysis with the application of estimating the size of hard to reach populations
• Variance Regularization for Accelerating Stochastic Optimization
• Why to "grow" and "harvest" deep learning models?
• Incorporating Chinese Radicals Into Neural Machine Translation: Deeper Than Character Level
• Tight bounds on the mutual coherence of sensing matrices for Wigner D-functions on regular grids
• Binary Search and First Order Gradient Based Method for Stochastic Optimization
• ManiGen: A Manifold Aided Black-box Generator of Adversarial Examples
• C2G-Net: Exploiting Morphological Properties for Image Classification
• A deep learning framework for segmentation of retinal layers from OCT images
• Breaking the Conventional Forward-Backward Tie in Neural Networks: Activation Functions
• Understanding Human Judgments of Causality
• The Effect of Optimization Methods on the Robustness of Out-of-Distribution Detection Approaches
• Chinese Poetry Generation with Flexible Styles
• Reparametrizing gradient descent
• Recurrent Neural Networks in Linguistic Theory: Revisiting Pinker and Prince (1988) and the Past Tense Debate