Recurrent Neural Network Regularization

Abstract

We present a simple regularization technique for Recurrent Neural Networks (RNNs) with Long Short-Term Memory (LSTM) units. Dropout, the most successful technique for regularizing neural networks, does not work well with RNNs and LSTMs. In this paper, we show how to correctly apply dropout to LSTMs, and show that it substantially reduces overfitting on a variety of tasks. These tasks include language modeling, speech recognition, image caption generation, and machine translation.

References in corpus (6)

• Sequence to Sequence Learning with Neural Networks
• Exploiting Similarities among Languages for Machine Translation
• Going Deeper with Convolutions
• How to Construct Deep Recurrent Neural Networks
• On Fast Dropout and its Applicability to Recurrent Networks
• Regularization and nonlinearities for neural language models: when are they needed?

Cited by in corpus (453)

• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• LSTM: A Search Space Odyssey
• Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network
• Neural Architecture Search with Reinforcement Learning
• Unsupervised Learning of Video Representations using LSTMs
• DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG
• Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
• A trans-disciplinary review of deep learning research for water resources scientists
• Character-Aware Neural Language Models
• Matching Networks for One Shot Learning
• Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challenge
• All One Needs to Know about Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agenda
• Exploring the Limits of Language Modeling
• A Theoretically Grounded Application of Dropout in Recurrent Neural Networks
• End-To-End Memory Networks
• Reasoning about Entailment with Neural Attention
• Efficient Neural Architecture Search via Parameter Sharing
• SNIP: Single-shot Network Pruning based on Connection Sensitivity
• Learning to Diagnose with LSTM Recurrent Neural Networks
• FPGA-based Accelerators of Deep Learning Networks for Learning and Classification: A Review
• An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data
• Pointer Sentinel Mixture Models
• Semi-supervised Sequence Learning
• Regularizing and Optimizing LSTM Language Models
• Scalable Bayesian Optimization Using Deep Neural Networks
• Predicting Process Behaviour using Deep Learning
• Born Again Neural Networks
• Gated Feedback Recurrent Neural Networks
• Action Recognition using Visual Attention
• Train faster, generalize better: Stability of stochastic gradient descent
• Quasi-Recurrent Neural Networks
• Learning to Execute
• Prolongation of SMAP to Spatio-temporally Seamless Coverage of Continental US Using a Deep Learning Neural Network
• Co-occurrence Feature Learning for Skeleton based Action Recognition using Regularized Deep LSTM Networks
• Device Placement Optimization with Reinforcement Learning
• A Comprehensive Review of Computer Vision in Sports: Open Issues, Future Trends and Research Directions
• Learning Longer Memory in Recurrent Neural Networks
• Qualitatively characterizing neural network optimization problems
• Describing Videos by Exploiting Temporal Structure
• A Survey of mmWave-based Human Sensing: Technology, Platform and Applications
• Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches
• Super-resolution-based Change Detection Network with Stacked Attention Module for Images with Different Resolutions
• Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling
• Data Noising as Smoothing in Neural Network Language Models
• Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations
• Multi-Sensor Prognostics using an Unsupervised Health Index based on LSTM Encoder-Decoder
• Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders
• Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation
• Machine Learning-Enabled IoT Security: Open Issues and Challenges Under Advanced Persistent Threats
• Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent Neural Networks
• Improving Neural Network Quantization without Retraining using Outlier Channel Splitting
• Deep Visual-Semantic Alignments for Generating Image Descriptions
• Beyond Data and Model Parallelism for Deep Neural Networks
• Bayesian Recurrent Neural Networks
• Machine-learning-assisted correction of correlated qubit errors in a topological code
• TVM: An Automated End-to-End Optimizing Compiler for Deep Learning
• Deep Learning for Computational Cytology: A Survey
• Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems
• Mixed-precision deep learning based on computational memory
• Learning Intrinsic Sparse Structures within Long Short-Term Memory
• Language to Logical Form with Neural Attention
• Variable Rate Image Compression with Recurrent Neural Networks
• Recurrent Environment Simulators
• Sequence-to-Sequence Learning as Beam-Search Optimization
• TimeNet: Pre-trained deep recurrent neural network for time series classification
• A Neural Knowledge Language Model
• Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences
• Generating News Headlines with Recurrent Neural Networks
• Addressing the Rare Word Problem in Neural Machine Translation
• Nonlinear Systems Identification Using Deep Dynamic Neural Networks
• RUDDER: Return Decomposition for Delayed Rewards
• Entropy-SGD: Biasing Gradient Descent Into Wide Valleys
• DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing
• Adversarial Generation of Natural Language
• Factorization tricks for LSTM networks
• FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network
• Natural Language Statistical Features of LSTM-generated Texts
• Recurrent Dropout without Memory Loss
• CoaCor: Code Annotation for Code Retrieval with Reinforcement Learning
• Recent advances for quantum classifiers
• An End-to-End Trainable Neural Network Model with Belief Tracking for Task-Oriented Dialog
• LSTM-Based System-Call Language Modeling and Robust Ensemble Method for Designing Host-Based Intrusion Detection Systems
• Analyzing Information Leakage of Updates to Natural Language Models
• Recurrent Neural Networks Hardware Implementation on FPGA
• EEGDnet: Fusing Non-Local and Local Self-Similarity for 1-D EEG Signal Denoising with 2-D Transformer
• DeepCare: A Deep Dynamic Memory Model for Predictive Medicine
• Noisy Activation Functions
• Knowledge Query Network: How Knowledge Interacts with Skills
• Hierarchical Attention Network for Action Recognition in Videos
• Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
• Text-based LSTM networks for Automatic Music Composition
• LFADS - Latent Factor Analysis via Dynamical Systems
• Gender Bias in Neural Natural Language Processing
• Benchmarking State-of-the-Art Deep Learning Software Tools
• Feedforward Sequential Memory Networks: A New Structure to Learn Long-term Dependency
• Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling
• Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback
• A System for Massively Parallel Hyperparameter Tuning
• Effective Quantization Methods for Recurrent Neural Networks
• A Radar Signal Deinterleaving Method Based on Semantic Segmentation with Neural Network
• Breaking the Softmax Bottleneck: A High-Rank RNN Language Model
• Physics-informed neural networks method in high-dimensional integrable systems
• Dynamic Evaluation of Neural Sequence Models
• A Deep Learning Approach for Forecasting Air Pollution in South Korea Using LSTM
• Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Exploration
• Effective and Efficient Dropout for Deep Convolutional Neural Networks
• Training LSTM Networks with Resistive Cross-Point Devices
• Tree-structured composition in neural networks without tree-structured architectures
• Structured Sequence Modeling with Graph Convolutional Recurrent Networks
• Biased Importance Sampling for Deep Neural Network Training
• Survey of Dropout Methods for Deep Neural Networks
• Dual Rectified Linear Units (DReLUs): A Replacement for Tanh Activation Functions in Quasi-Recurrent Neural Networks
• Augmenting Self-attention with Persistent Memory
• Dynamic Occupancy Grid Mapping with Recurrent Neural Networks
• Recurrent Neural Radio Anomaly Detection
• A Survey of Optimization Methods from a Machine Learning Perspective
• Toward Trainability of Quantum Neural Networks
• Learning Global Features for Coreference Resolution
• Compression of Recurrent Neural Networks for Efficient Language Modeling
• Gated Word-Character Recurrent Language Model
• Revisiting Activation Regularization for Language RNNs
• LightRNN: Memory and Computation-Efficient Recurrent Neural Networks
• Combining the Best of Convolutional Layers and Recurrent Layers: A Hybrid Network for Semantic Segmentation
• Segmental Recurrent Neural Networks for End-to-end Speech Recognition
• Uncovering Temporal Context for Video Question and Answering
• Fast-Slow Recurrent Neural Networks
• Higher Order Recurrent Neural Networks
• Learning Neural Markers of Schizophrenia Disorder Using Recurrent Neural Networks
• DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework
• A Survey of FPGA Based Deep Learning Accelerators: Challenges and Opportunities
• Towards Binary-Valued Gates for Robust LSTM Training
• Evaluating prose style transfer with the Bible
• Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks
• Adversarial Generation of Training Examples: Applications to Moving Vehicle License Plate Recognition
• Hierarchical Multitask Learning for CTC-based Speech Recognition
• A Neural Transducer
• Latent Attention For If-Then Program Synthesis
• Neural Networks Compression for Language Modeling
• Spatio-temporal Stacked LSTM for Temperature Prediction in Weather Forecasting
• Neural Text Generation: A Practical Guide
• A Tutorial on Deep Latent Variable Models of Natural Language
• Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
• Extracting Parallel Sentences with Bidirectional Recurrent Neural Networks to Improve Machine Translation
• Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining
• pix2code: Generating Code from a Graphical User Interface Screenshot
• Recurrent Additive Networks
• End-to-End Neural Segmental Models for Speech Recognition
• Comparing Neural- and N-Gram-Based Language Models for Word Segmentation
• The Implicit and Explicit Regularization Effects of Dropout
• CFO: Conditional Focused Neural Question Answering with Large-scale Knowledge Bases
• rnn : Recurrent Library for Torch
• Weakly Supervised Dense Video Captioning
• Regularizing RNNs by Stabilizing Activations
• Regularizing RNNs for Caption Generation by Reconstructing The Past with The Present
• Natural Language Generation in Dialogue using Lexicalized and Delexicalized Data
• Shape Inpainting using 3D Generative Adversarial Network and Recurrent Convolutional Networks
• Predicting Auction Price of Vehicle License Plate with Deep Recurrent Neural Network
• Multilingual Language Processing From Bytes
• Run-Time Efficient RNN Compression for Inference on Edge Devices
• A recurrent neural network without chaos
• Motion Estimation in Occupancy Grid Maps in Stationary Settings Using Recurrent Neural Networks
• Themis: Fair and Efficient GPU Cluster Scheduling
• Textual Entailment with Structured Attentions and Composition
• A Neural Architecture for Generating Natural Language Descriptions from Source Code Changes
• Deriving Neural Architectures from Sequence and Graph Kernels
• Neural Language Modeling by Jointly Learning Syntax and Lexicon
• Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation
• What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignment
• Joint Online Spoken Language Understanding and Language Modeling with Recurrent Neural Networks
• You May Not Need Attention
• Model Slicing for Supporting Complex Analytics with Elastic Inference Cost and Resource Constraints
• Leveraging Sentence-level Information with Encoder LSTM for Semantic Slot Filling
• Affect-LM: A Neural Language Model for Customizable Affective Text Generation
• Adversarial Learning for Improved Onsets and Frames Music Transcription
• DeepTwist: Learning Model Compression via Occasional Weight Distortion
• Sigsoftmax: Reanalysis of the Softmax Bottleneck
• Call Attention to Rumors: Deep Attention Based Recurrent Neural Networks for Early Rumor Detection
• Noisin: Unbiased Regularization for Recurrent Neural Networks
• Domain Adaptive Dialog Generation via Meta Learning
• Implicit Dimension Identification in User-Generated Text with LSTM Networks
• SPASS: Scientific Prominence Active Search System with Deep Image Captioning Network
• Wisture: RNN-based Learning of Wireless Signals for Gesture Recognition in Unmodified Smartphones
• Stragglers Are Not Disaster: A Hybrid Federated Learning Algorithm with Delayed Gradients
• Semantic Compositional Networks for Visual Captioning
• Consensus-based Sequence Training for Video Captioning
• Stabilizing Gradients for Deep Neural Networks via Efficient SVD Parameterization
• Zero-Shot Dialog Generation with Cross-Domain Latent Actions
• Character-Word LSTM Language Models
• Deep Learning in Mining Biological Data
• Domain Control for Neural Machine Translation
• MUST-CNN: A Multilayer Shift-and-Stitch Deep Convolutional Architecture for Sequence-based Protein Structure Prediction
• The Statistical Recurrent Unit
• Neural Morphological Tagging from Characters for Morphologically Rich Languages
• Unsupervised Pretraining for Sequence to Sequence Learning
• Selfish Sparse RNN Training
• Grow and Prune Compact, Fast, and Accurate LSTMs
• Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
• LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks
• Strongly-Typed Recurrent Neural Networks
• Quantifying Uncertainties in Natural Language Processing Tasks
• Convolutional Recurrent Neural Networks for Glucose Prediction
• A Comparative Review of Recent Few-Shot Object Detection Algorithms
• Modelling Student Behavior using Granular Large Scale Action Data from a MOOC
• Recurrent Convolutional Networks for Pulmonary Nodule Detection in CT Imaging
• Cross Domain Knowledge Transfer for Person Re-identification
• Enriching Rare Word Representations in Neural Language Models by Embedding Matrix Augmentation
• Restricted Recurrent Neural Networks
• Scheduling Computation Graphs of Deep Learning Models on Manycore CPUs
• Tracking Human-like Natural Motion Using Deep Recurrent Neural Networks
• Bayesian Sparsification of Recurrent Neural Networks
• Optimal regularizations for data generation with probabilistic graphical models
• Generative Encoder-Decoder Models for Task-Oriented Spoken Dialog Systems with Chatting Capability
• Sequence Labeling: A Practical Approach
• Lie Access Neural Turing Machine
• Phase of Flight Classification in Aviation Safety using LSTM, GRU, and BiLSTM: A Case Study with ASN Dataset
• rTop-k: A Statistical Estimation Approach to Distributed SGD
• Retraining-Based Iterative Weight Quantization for Deep Neural Networks
• Pushing the bounds of dropout
• Deep Neural Machine Translation with Weakly-Recurrent Units
• Interacting Attention-gated Recurrent Networks for Recommendation
• Multitask Learning with CTC and Segmental CRF for Speech Recognition
• Learning K-way D-dimensional Discrete Code For Compact Embedding Representations
• Dynamic Entity Representations in Neural Language Models
• A detailed comparative study of open source deep learning frameworks
• Sentence Simplification with Deep Reinforcement Learning
• Converting Your Thoughts to Texts: Enabling Brain Typing via Deep Feature Learning of EEG Signals
• State-Regularized Recurrent Neural Networks
• Precision Highway for Ultra Low-Precision Quantization
• Visual-Textual Association with Hardest and Semi-Hard Negative Pairs Mining for Person Search
• Semantic Variation in Online Communities of Practice
• Low-Resource Language Modelling of South African Languages
• Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization
• Neural Latent Extractive Document Summarization
• Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learning
• Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiency
• Improving Neural Network with Uniform Sparse Connectivity
• Skeleton-based Gait Index Estimation with LSTMs
• Growing Efficient Deep Networks by Structured Continuous Sparsification
• Distributed Training Large-Scale Deep Architectures
• Self-Normalization Properties of Language Modeling
• Faster Neural Network Training with Approximate Tensor Operations
• Learning a Prior over Intent via Meta-Inverse Reinforcement Learning
• Breaking the Activation Function Bottleneck through Adaptive Parameterization
• Performance Evaluation of Deep Learning Tools in Docker Containers
• Obstacle-Transformer: A Trajectory Prediction Network Based on Surrounding Trajectories
• Multimodal Memory Modelling for Video Captioning
• Cross-Modal Retrieval with Implicit Concept Association
• A Simple LSTM model for Transition-based Dependency Parsing
• Hardware-Guided Symbiotic Training for Compact, Accurate, yet Execution-Efficient LSTM
• Proactive Resource Management for LTE in Unlicensed Spectrum: A Deep Learning Perspective
• A Statistical Investigation of Long Memory in Language and Music
• Efficient batchwise dropout training using submatrices
• Variational Bi-LSTMs
• On the Effective Use of Pretraining for Natural Language Inference
• A Convergence Analysis for A Class of Practical Variance-Reduction Stochastic Gradient MCMC
• Temporally Coherent Full 3D Mesh Human Pose Recovery from Monocular Video
• Evaluating Models of Robust Word Recognition with Serial Reproduction
• Investigation on N-gram Approximated RNNLMs for Recognition of Morphologically Rich Speech
• Unsupervised and Efficient Vocabulary Expansion for Recurrent Neural Network Language Models in ASR
• Structurally Sparsified Backward Propagation for Faster Long Short-Term Memory Training
• Sentence Simplification with Memory-Augmented Neural Networks
• Training Deep Networks without Learning Rates Through Coin Betting
• Molecular De Novo Design through Deep Reinforcement Learning
• Dance Dance Convolution
• LSTM-Based Predictions for Proactive Information Retrieval
• A Survey of Spatio-Temporal EEG data Analysis: from Models to Applications
• Driving with Data: Modeling and Forecasting Vehicle Fleet Maintenance in Detroit
• Efficient Decoupled Neural Architecture Search by Structure and Operation Sampling
• Noisy Recurrent Neural Networks
• Residual Matrix Product State for Machine Learning
• Language Modeling with Sparse Product of Sememe Experts
• An Attention Mechanism for Answer Selection Using a Combined Global and Local View
• Bilingual-GAN: A Step Towards Parallel Text Generation
• Low-rank passthrough neural networks
• Scalable Bayesian Learning of Recurrent Neural Networks for Language Modeling
• End-to-end Image Captioning Exploits Multimodal Distributional Similarity
• Almost Sure Convergence of Dropout Algorithms for Neural Networks
• Delving Deeper into the Decoder for Video Captioning
• Intent Recognition in Smart Living Through Deep Recurrent Neural Networks
• State-of-the-art Chinese Word Segmentation with Bi-LSTMs
• Bayesian Methods for Semi-supervised Text Annotation
• On the Complementarity of Data Selection and Fine Tuning for Domain Adaptation
• Echo: Compiler-based GPU Memory Footprint Reduction for LSTM RNN Training
• Going Wider: Recurrent Neural Network With Parallel Cells
• PAPooling: Graph-based Position Adaptive Aggregation of Local Geometry in Point Clouds
• Attentive batch normalization for lstm-based acoustic modeling of speech recognition
• An Empirical Study on Hyperparameter Optimization for Fine-Tuning Pre-trained Language Models
• Rapid detection of phase transitions from Monte Carlo samples before equilibrium
• Parameter Re-Initialization through Cyclical Batch Size Schedules
• Language Models with Pre-Trained (GloVe) Word Embeddings
• Multilingual End-to-End Speech Translation
• Online Sequence Training of Recurrent Neural Networks with Connectionist Temporal Classification
• Online Deception Detection Refueled by Real World Data Collection
• Recurrent Memory Array Structures
• Benchmarking Deep Sequential Models on Volatility Predictions for Financial Time Series
• S-SGD: Symmetrical Stochastic Gradient Descent with Weight Noise Injection for Reaching Flat Minima
• Surprisal-Triggered Conditional Computation with Neural Networks
• Dependency Parsing as Head Selection
• Avoiding overfitting of multilayer perceptrons by training derivatives
• Teaching Machines to Converse
• Optimization Algorithm for Feedback and Feedforward Policies towards Robot Control Robust to Sensing Failures
• Log Message Anomaly Detection and Classification Using Auto-B/LSTM and Auto-GRU
• Compressing Language Models using Doped Kronecker Products
• MNIST-NET10: A heterogeneous deep networks fusion based on the degree of certainty to reach 0.1 error rate. Ensembles overview and proposal
• Orthant Based Proximal Stochastic Gradient Method for ${\ell }_1$-Regularized Optimization
• Detecting and Exorcising Statistical Demons from Language Models with Anti-Models of Negative Data
• An Empirical Study of Excitation and Aggregation Design Adaptions in CLIP4Clip for Video-Text Retrieval
• Comprehending Real Numbers: Development of Bengali Real Number Speech Corpus
• Query2Vec: An Evaluation of NLP Techniques for Generalized Workload Analytics
• Teaching Machines to Code: Neural Markup Generation with Visual Attention
• PIN: A Novel Parallel Interactive Network for Spoken Language Understanding
• Parsing Speech: A Neural Approach to Integrating Lexical and Acoustic-Prosodic Information
• emLam -- a Hungarian Language Modeling baseline
• Restoring ancient text using deep learning: a case study on Greek epigraphy
• Visual Fashion-Product Search at SK Planet
• Team EP at TAC 2018: Automating data extraction in systematic reviews of environmental agents
• Ising-Dropout: A Regularization Method for Training and Compression of Deep Neural Networks
• Learning from Videos with Deep Convolutional LSTM Networks
• Model Interpolation with Trans-dimensional Random Field Language Models for Speech Recognition
• Wind Estimation Using Quadcopter Motion: A Machine Learning Approach
• Early Improving Recurrent Elastic Highway Network
• Modular Universal Reparameterization: Deep Multi-task Learning Across Diverse Domains
• Improving Context Aware Language Models
• A Decoupled Learning Strategy for Massive Access Optimization in Cellular IoT Networks
• Artificial Intelligence based Autonomous Molecular Design for Medical Therapeutic: A Perspective
• BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?
• Learning Less-Overlapping Representations
• Learning Neuron Non-Linearities with Kernel-Based Deep Neural Networks
• Energy-Based Models with Applications to Speech and Language Processing
• Using LSTMs to Model the Java Programming Language
• Evolving Neural Selection with Adaptive Regularization
• Putting visual object recognition in context
• Predicting Opioid Use Disorder from Longitudinal Healthcare Data using Multi-stream Transformer
• A Structured Distributional Model of Sentence Meaning and Processing
• WeNet: Weighted Networks for Recurrent Network Architecture Search
• Diabetes Mellitus Forecasting Using Population Health Data in Ontario, Canada
• Learning Temporal Embeddings for Complex Video Analysis
• Refined Gate: A Simple and Effective Gating Mechanism for Recurrent Units
• MTLE: A Multitask Learning Encoder of Visual Feature Representations for Video and Movie Description
• Driving with Data in the Motor City: Mining and Modeling Vehicle Fleet Maintenance Data
• Syntax-Aware Language Modeling with Recurrent Neural Networks
• Learning Recurrent Binary/Ternary Weights
• Modeling the spatio-temporal dynamics of land use change with recurrent neural networks
• AntMan: Sparse Low-Rank Compression to Accelerate RNN inference
• Direct Output Connection for a High-Rank Language Model
• Automated Radiological Report Generation For Chest X-Rays With Weakly-Supervised End-to-End Deep Learning
• Character n-gram Embeddings to Improve RNN Language Models
• Improving Text Generation with Student-Forcing Optimal Transport
• Brittle AI, Causal Confusion, and Bad Mental Models: Challenges and Successes in the XAI Program
• Active Screening for Recurrent Diseases: A Reinforcement Learning Approach
• Action Search: Spotting Actions in Videos and Its Application to Temporal Action Localization
• Sarcasm Analysis using Conversation Context
• Long Short-Term Memory with Dynamic Skip Connections
• Deep Learning for Stock Selection Based on High Frequency Price-Volume Data
• DARB: A Density-Aware Regular-Block Pruning for Deep Neural Networks
• Regularizing Recurrent Networks - On Injected Noise and Norm-based Methods
• Learning On-Road Visual Control for Self-Driving Vehicles with Auxiliary Tasks
• Doubly Sparse: Sparse Mixture of Sparse Experts for Efficient Softmax Inference
• Behavior Gated Language Models
• Exploring Weight Symmetry in Deep Neural Networks
• Dependency Parsing with LSTMs: An Empirical Evaluation
• An ETF view of Dropout regularization
• Stochastic Gradient Methods with Block Diagonal Matrix Adaptation
• Listen, Attend, Spell and Adapt: Speaker Adapted Sequence-to-Sequence ASR
• Tackling Sequence to Sequence Mapping Problems with Neural Networks
• Skeleton-based Hand-Gesture Recognition with Lightweight Graph Convolutional Networks
• Few-Shot Object Recognition from Machine-Labeled Web Images
• Representation and Correlation Enhanced Encoder-Decoder Framework for Scene Text Recognition
• Contrast-reconstruction Representation Learning for Self-supervised Skeleton-based Action Recognition
• Learning sentence embeddings using Recursive Networks
• Rational Recurrences
• PMI Matrix Approximations with Applications to Neural Language Modeling
• Image Super-Resolution via Attention based Back Projection Networks
• Learning Simpler Language Models with the Differential State Framework
• Accelerated Reinforcement Learning for Sentence Generation by Vocabulary Prediction
• Dynamics Estimation Using Recurrent Neural Network
• Cavs: A Vertex-centric Programming Interface for Dynamic Neural Networks
• Behind the Smile: Mental Health Implications of Mother-Infant Interactions Revealed Through Smile Analysis
• Joint Spatial and Layer Attention for Convolutional Networks
• Empirical Evaluation of A New Approach to Simplifying Long Short-term Memory (LSTM)
• Pouring Dynamics Estimation Using Gated Recurrent Units
• Sampling-based Gradient Regularization for Capturing Long-Term Dependencies in Recurrent Neural Networks
• Sampling-free Uncertainty Estimation in Gated Recurrent Units with Exponential Families
• Improved training of neural trans-dimensional random field language models with dynamic noise-contrastive estimation
• Knowledge Distillation For Recurrent Neural Network Language Modeling With Trust Regularization
• Lift-the-flap: what, where and when for context reasoning
• Generate Image Descriptions based on Deep RNN and Memory Cells for Images Features
• Adversarial Dropout for Recurrent Neural Networks
• Continuous-Time Systems for Solving 0-1 Integer Linear Programming Feasibility Problems
• Online Supervised Learning for Traffic Load Prediction in Framed-ALOHA Networks
• Blockwise Adaptivity: Faster Training and Better Generalization in Deep Learning
• Are All Languages Equally Hard to Language-Model?
• Spectral Pruning for Recurrent Neural Networks
• Persistence pays off: Paying Attention to What the LSTM Gating Mechanism Persists
• Variational Inference-Based Dropout in Recurrent Neural Networks for Slot Filling in Spoken Language Understanding
• LSTM recurrent neural network assisted aircraft stall prediction for enhanced situational awareness
• Language Modeling with Highway LSTM
• Doping: A technique for efficient compression of LSTM models using sparse structured additive matrices
• Machine Learning Algorithms for Financial Asset Price Forecasting
• Analysing Dropout and Compounding Errors in Neural Language Models
• Optimizing and Contrasting Recurrent Neural Network Architectures
• End-to-end training of deep kernel map networks for image classification
• Large-Dimensional Random Matrix Theory and Its Applications in Deep Learning and Wireless Communications
• Robot Gaining Accurate Pouring Skills through Self-Supervised Learning and Generalization
• Automatic Extraction of Commonsense LocatedNear Knowledge
• A Generalized Language Model in Tensor Space
• Adaptive Noise Injection: A Structure-Expanding Regularization for RNN
• Structured in Space, Randomized in Time: Leveraging Dropout in RNNs for Efficient Training
• IC Neuron: An Efficient Unit to Construct Neural Networks
• All Word Embeddings from One Embedding
• Investigating Alternatives to the Root Mean Square for Adaptive Gradient Methods
• Regularizing Recurrent Neural Networks via Sequence Mixup
• Dialog Context Language Modeling with Recurrent Neural Networks
• Revisiting Simple Neural Probabilistic Language Models
• Deep Learning for Latent Events Forecasting in Twitter Aided Caching Networks
• Autoencoding Undirected Molecular Graphs With Neural Networks
• Evaluating Temporal Patterns in Applied Infant Affect Recognition
• Adaptive Recurrent Neural Network Based on Mixture Layer
• Input-to-Output Gate to Improve RNN Language Models
• Modulating Regularization Frequency for Efficient Compression-Aware Model Training
• Iterative evaluation of LSTM cells
• De-identification In practice
• Linear Additive Markov Processes
• Canoe : A System for Collaborative Learning for Neural Nets
• Efficient Segmental Cascades for Speech Recognition
• Large Margin Neural Language Model
• Restricted Recurrent Neural Tensor Networks: Exploiting Word Frequency and Compositionality
• Explicit Spatiotemporal Joint Relation Learning for Tracking Human Pose
• Quantal synaptic dilution enhances sparse encoding and dropout regularisation in deep networks
• Neural Language Modeling With Implicit Cache Pointers
• Dropout during inference as a model for neurological degeneration in an image captioning network
• Recurrent Models for Situation Recognition
• Automatic Construction of Sememe Knowledge Bases via Dictionaries
• Relational Weight Priors in Neural Networks for Abstract Pattern Learning and Language Modelling
• Modeling Social Organizations as Communication Networks
• Leveraging Recurrent Neural Networks for Multimodal Recognition of Social Norm Violation in Dialog
• Learning Sparse Structured Ensembles with SG-MCMC and Network Pruning
• Language modeling with Neural trans-dimensional random fields
• Learning neural trans-dimensional random field language models with noise-contrastive estimation
• RotationOut as a Regularization Method for Neural Network
• Thick-Net: Parallel Network Structure for Sequential Modeling
• Predictive Process Model Monitoring using Recurrent Neural Networks
• Graduate Employment Prediction with Bias
• Pastprop-RNN: improved predictions of the future by correcting the past
• An Embedded Deep Learning based Word Prediction
• Improve Lexicon-based Word Embeddings By Word Sense Disambiguation
• Alleviating Overfitting for Polysemous Words for Word Representation Estimation Using Lexicons
• Modality Compensation Network: Cross-Modal Adaptation for Action Recognition
• Distinctive-attribute Extraction for Image Captioning
• Discourse Embellishment Using a Deep Encoder-Decoder Network
• Transferable Graph Optimizers for ML Compilers
• Syllable-level Neural Language Model for Agglutinative Language
• Patterns versus Characters in Subword-aware Neural Language Modeling