LSTM: A Search Space Odyssey

Abstract

Several variants of the Long Short-Term Memory (LSTM) architecture for recurrent neural networks have been proposed since its inception in 1995. In recent years, these networks have become the state-of-the-art models for a variety of machine learning problems. This has led to a renewed interest in understanding the role and utility of various computational components of typical LSTM variants. In this paper, we present the first large-scale analysis of eight LSTM variants on three representative tasks: speech recognition, handwriting recognition, and polyphonic music modeling. The hyperparameters of all LSTM variants for each task were optimized separately using random search, and their importance was assessed using the powerful fANOVA framework. In total, we summarize the results of 5400 experimental runs ($\approx 15$ years of CPU time), which makes our study the largest of its kind on LSTM networks. Our results show that none of the variants can improve upon the standard LSTM architecture significantly, and demonstrate the forget gate and the output activation function to be its most critical components. We further observe that the studied hyperparameters are virtually independent and derive guidelines for their efficient adjustment.

12 pages, 6 figures

References in corpus (6)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Practical Bayesian Optimization of Machine Learning Algorithms
• Recurrent Neural Network Regularization
• Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription
• Addressing the Rare Word Problem in Neural Machine Translation
• Protein Secondary Structure Prediction with Long Short Term Memory Networks

Cited by in corpus (433)

• Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• Character-level Convolutional Networks for Text Classification
• Natural Language Processing: State of The Art, Current Trends and Challenges
• Neural Architecture Search: A Survey
• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• Federated Learning for Mobile Keyboard Prediction
• Deep Neural Networks and Tabular Data: A Survey
• Visualizing and Understanding Recurrent Networks
• ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases
• A Survey and Critique of Multiagent Deep Reinforcement Learning
• Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregation
• Semi-supervised Sequence Learning
• Understanding Neural Networks through Representation Erasure
• Predicting Process Behaviour using Deep Learning
• Light Gated Recurrent Units for Speech Recognition
• Deep Bidirectional and Unidirectional LSTM Recurrent Neural Network for Network-wide Traffic Speed Prediction
• Unsupervised and Semi-supervised Anomaly Detection with LSTM Neural Networks
• Deep, Convolutional, and Recurrent Models for Human Activity Recognition using Wearables
• A Comprehensive Analysis of Deep Regression
• Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
• Early Stage Malware Prediction Using Recurrent Neural Networks
• Mixed Neural Network Approach for Temporal Sleep Stage Classification
• Land Cover Classification via Multi-temporal Spatial Data by Recurrent Neural Networks
• Grid Long Short-Term Memory
• Rainfall-Runoff Prediction at Multiple Timescales with a Single Long Short-Term Memory Network
• Working Memory Connections for LSTM
• An overview and comparative analysis of Recurrent Neural Networks for Short Term Load Forecasting
• Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
• Temporal Attention augmented Bilinear Network for Financial Time-Series Data Analysis
• An Analysis of Hierarchical Text Classification Using Word Embeddings
• Crop Yield Prediction Integrating Genotype and Weather Variables Using Deep Learning
• Neural GPUs Learn Algorithms
• Deep Neural Networks with Koopman Operators for Modeling and Control of Autonomous Vehicles
• Classification of Hand Movements from EEG using a Deep Attention-based LSTM Network
• A Biologically Plausible Supervised Learning Method for Spiking Neural Networks Using the Symmetric STDP Rule
• Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges
• On Extended Long Short-term Memory and Dependent Bidirectional Recurrent Neural Network
• Jet Flavor Classification in High-Energy Physics with Deep Neural Networks
• A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations
• Language Modeling with Deep Transformers
• A comparison of LSTM and GRU networks for learning symbolic sequences
• A Comprehensive Study of Deep Bidirectional LSTM RNNs for Acoustic Modeling in Speech Recognition
• Deep-learning-based surrogate flow modeling and geological parameterization for data assimilation in 3D subsurface flow
• Learning in Memristive Neural Network Architectures using Analog Backpropagation Circuits
• Deep learning incorporating biologically-inspired neural dynamics
• Anomaly Detection in Univariate Time-series: A Survey on the State-of-the-Art
• Time series forecasting of new cases and new deaths rate for COVID-19 using deep learning methods
• HiPPO: Recurrent Memory with Optimal Polynomial Projections
• Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation
• Deep Learning for Procedural Content Generation
• Training recurrent neural networks robust to incomplete data: application to Alzheimer's disease progression modeling
• Model-Free Quantum Control with Reinforcement Learning
• Applications of deep learning in traffic congestion detection, prediction and alleviation: A survey
• Generation and Comprehension of Unambiguous Object Descriptions
• Learning Recurrent Span Representations for Extractive Question Answering
• On the Feasibility of Transfer-learning Code Smells using Deep Learning
• Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN
• Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks
• Neural Machine Translation and Sequence-to-sequence Models: A Tutorial
• Pretrained Domain-Specific Language Model for General Information Retrieval Tasks in the AEC Domain
• A Deep Evolutionary Approach to Bioinspired Classifier Optimisation for Brain-Machine Interaction
• Recent Advances in Deep Learning: An Overview
• Towards Universal Paraphrastic Sentence Embeddings
• Architectural Complexity Measures of Recurrent Neural Networks
• Over a Decade of Social Opinion Mining: A Systematic Review
• Review: Deep Learning in Electron Microscopy
• Deep Learning for Short-Term Voltage Stability Assessment of Power Systems
• RUDDER: Return Decomposition for Delayed Rewards
• DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing
• Federated Learning Of Out-Of-Vocabulary Words
• Natural Language Statistical Features of LSTM-generated Texts
• Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial
• SAR Target Recognition Using the Multi-aspect-aware Bidirectional LSTM Recurrent Neural Networks
• Trellis Networks for Sequence Modeling
• Capsule Attention for Multimodal EEG-EOG Representation Learning with Application to Driver Vigilance Estimation
• Time Series Forecasting Using LSTM Networks: A Symbolic Approach
• Using a Recurrent Neural Network to Reconstruct Quantum Dynamics of a Superconducting Qubit from Physical Observations
• Machine Learning for Condensed Matter Physics
• A non-intrusive reduced order modeling framework for quasi-geostrophic turbulence
• On the Origin of Deep Learning
• Deep Learning for Steganalysis of Diverse Data Types: A review of methods, taxonomy, challenges and future directions
• Self-supervised Autoregressive Domain Adaptation for Time Series Data
• Recurrent Neural Networks Hardware Implementation on FPGA
• Controlling Steering Angle for Cooperative Self-driving Vehicles utilizing CNN and LSTM-based Deep Networks
• Deep Sequential Segmentation of Organs in Volumetric Medical Scans
• Deep Recurrent Neural Networks for ECG Signal Denoising
• Using LSTM recurrent neural networks for monitoring the LHC superconducting magnets
• Human Activity Recognition Based on Wearable Sensor Data: A Standardization of the State-of-the-Art
• Towards Real-Time Respiratory Motion Prediction based on Long Short-Term Memory Neural Networks
• Neural Memory Networks for Seizure Type Classification
• Recurrent Neural Networks for Time Series Forecasting
• Biologically inspired alternatives to backpropagation through time for learning in recurrent neural nets
• Dynamic Mortality Risk Predictions in Pediatric Critical Care Using Recurrent Neural Networks
• Explaining and Interpreting LSTMs
• Subsurface structure analysis using computational interpretation and learning: A visual signal processing perspective
• Guiding Long-Short Term Memory for Image Caption Generation
• The unreasonable effectiveness of the forget gate
• Collaborative Filtering with Recurrent Neural Networks
• Improved Transition-Based Parsing by Modeling Characters instead of Words with LSTMs
• Using Recurrent Neural Networks to Optimize Dynamical Decoupling for Quantum Memory
• Massive Exploration of Neural Machine Translation Architectures
• Generating Focussed Molecule Libraries for Drug Discovery with Recurrent Neural Networks
• Deep Sequential Feature Learning in Clinical Image Classification of Infectious Keratitis
• On the Potential of Simple Framewise Approaches to Piano Transcription
• What is the best RNN-cell structure to forecast each time series behavior?
• Improving performance of recurrent neural network with relu nonlinearity
• Accurate Real Time Localization Tracking in A Clinical Environment using Bluetooth Low Energy and Deep Learning
• Adaptive Learning Rule for Hardware-based Deep Neural Networks Using Electronic Synapse Devices
• A Siamese Long Short-Term Memory Architecture for Human Re-Identification
• Epileptic Seizure Detection: A Deep Learning Approach
• A neural network memory prefetcher using semantic locality
• OxIOD: The Dataset for Deep Inertial Odometry
• Optimizing Speech Recognition For The Edge
• Learning Long-Term Dependencies in Irregularly-Sampled Time Series
• Continuous Human Action Recognition for Human-Machine Interaction: A Review
• Mosaic Flows: A Transferable Deep Learning Framework for Solving PDEs on Unseen Domains
• AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search
• Modeling Context in Referring Expressions
• Landmark Tracking in Liver US images Using Cascade Convolutional Neural Networks with Long Short-Term Memory
• Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odyssey
• Natural Language Understanding with Distributed Representation
• Time2Vec: Learning a Vector Representation of Time
• A Survey on Deep Learning Methods for Robot Vision
• Identifying Emotions from Walking using Affective and Deep Features
• Deep Learning Based Packet Detection and Carrier Frequency Offset Estimation in IEEE 802.11ah
• Z-GCNETs: Time Zigzags at Graph Convolutional Networks for Time Series Forecasting
• Larger-Context Language Modelling
• Neural Information Retrieval: A Literature Review
• Generative Models for Effective ML on Private, Decentralized Datasets
• Robust and Subject-Independent Driving Manoeuvre Anticipation through Domain-Adversarial Recurrent Neural Networks
• Federated Learning Meets Natural Language Processing: A Survey
• A Survey of Visual Transformers
• Speech Prediction using an Adaptive Recurrent Neural Network with Application to Packet Loss Concealment
• Generative Choreography using Deep Learning
• Benchmarking of LSTM Networks
• Put your money where your mouth is: Using deep learning to identify consumer tribes from word usage
• DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning
• An unsupervised latent/output physics-informed convolutional-LSTM network for solving partial differential equations using peridynamic differential operator
• Mental Task Classification Using Electroencephalogram Signal
• Stacked Bidirectional and Unidirectional LSTM Recurrent Neural Network for Forecasting Network-wide Traffic State with Missing Values
• Lipreading with Long Short-Term Memory
• Implicit Language Model in LSTM for OCR
• Combining Visible Light and Infrared Imaging for Efficient Detection of Respiratory Infections such as COVID-19 on Portable Device
• Feature Engineering and Forecasting via Derivative-free Optimization and Ensemble of Sequence-to-sequence Networks with Applications in Renewable Energy
• An Experimental Study of LSTM Encoder-Decoder Model for Text Simplification
• A Framework and Call to Action for the Future Development of EMG-Based Input in HCI
• Deep Independently Recurrent Neural Network (IndRNN)
• Accuracy of neural networks for the simulation of chaotic dynamics: precision of training data vs precision of the algorithm
• A Taxonomy for Neural Memory Networks
• Augmentation Scheme for Dealing with Imbalanced Network Traffic Classification Using Deep Learning
• Empirical evaluation of shallow and deep learning classifiers for Arabic sentiment analysis
• Multi-modal gated recurrent units for image description
• Long-term Multi-granularity Deep Framework for Driver Drowsiness Detection
• Cortical microcircuits as gated-recurrent neural networks
• Improving the Gating Mechanism of Recurrent Neural Networks
• Time Series Forecasting Using Manifold Learning
• Artificial Intelligence for Cochlear Implants: Review of Strategies, Challenges, and Perspectives
• Deep Recurrent Architectures for Seismic Tomography
• Global Aggregations of Local Explanations for Black Box models
• Is it enough to optimize CNN architectures on ImageNet?
• IONet: Learning to Cure the Curse of Drift in Inertial Odometry
• Model-Size Reduction for Reservoir Computing by Concatenating Internal States Through Time
• Hyperspectral Image Denoising via Spatial-Spectral Recurrent Transformer
• Audio Word2Vec: Unsupervised Learning of Audio Segment Representations using Sequence-to-sequence Autoencoder
• PreCNet: Next-Frame Video Prediction Based on Predictive Coding
• Kronecker CP Decomposition with Fast Multiplication for Compressing RNNs
• Artificial intelligence for topic modelling in Hindu philosophy: mapping themes between the Upanishads and the Bhagavad Gita
• rnn : Recurrent Library for Torch
• Network Intrusion Detection based on LSTM and Feature Embedding
• Physics-informed generative neural network: an application to troposphere temperature prediction
• An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learning
• Understanding Hidden Memories of Recurrent Neural Networks
• Robustness of Neural Networks against Storage Media Errors
• Deep Reinforcement Learning-based Content Migration for Edge Content Delivery Networks with Vehicular Nodes
• Self-Explaining Structures Improve NLP Models
• NFDLM: A Lightweight Network Flow based Deep Learning Model for DDoS Attack Detection in IoT Domains
• Unsupervised Deep Anomaly Detection for Multi-Sensor Time-Series Signals
• Interpreting Deep Learning Models in Natural Language Processing: A Review
• Context-Aware Attention-Based Data Augmentation for POI Recommendation
• Effects of spatiotemporal correlations in wind data on neural network-based wind predictions
• Unsupervised Neural Domain Adaptation for Document Image Binarization
• Deriving Neural Architectures from Sequence and Graph Kernels
• Subregular Complexity and Deep Learning
• A Survey on Applications of Artificial Intelligence in Fighting Against COVID-19
• Recurrent Memory Networks for Language Modeling
• General Value Function Networks
• Large-Scale User Modeling with Recurrent Neural Networks for Music Discovery on Multiple Time Scales
• Analyzing and Exploiting NARX Recurrent Neural Networks for Long-Term Dependencies
• A deep learning pipeline for identification of motor units in musculoskeletal ultrasound
• Learning Compact Recurrent Neural Networks
• Machine Learning for Reliability Engineering and Safety Applications: Review of Current Status and Future Opportunities
• SNODE: Spectral Discretization of Neural ODEs for System Identification
• Expressive TTS Training with Frame and Style Reconstruction Loss
• Self-Supervised Learning for Enhancing Angular Resolution in Automotive MIMO Radars
• Basis Pursuit Denoising via Recurrent Neural Network Applied to Super-resolving SAR Tomography
• Learning to Recognize Actions on Objects in Egocentric Video with Attention Dictionaries
• DeepHealth: Review and challenges of artificial intelligence in health informatics
• E-cheating Prevention Measures: Detection of Cheating at Online Examinations Using Deep Learning Approach -- A Case Study
• Identifying tax evasion in Mexico with tools from network science and machine learning
• Watch What You Just Said: Image Captioning with Text-Conditional Attention
• Investigating gated recurrent neural networks for speech synthesis
• NaturalAE: Natural and Robust Physical Adversarial Examples for Object Detectors
• Federated Learning of N-gram Language Models
• Toward Visual Grounding: A Survey
• LSTM Neural Networks: Input to State Stability and Probabilistic Safety Verification
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
• Knowledge Infused Learning (K-IL): Towards Deep Incorporation of Knowledge in Deep Learning
• Scene Synthesis from Human Motion
• Discovering Bayesian Market Views for Intelligent Asset Allocation
• An empirical comparison of deep-neural-network architectures for next activity prediction using context-enriched process event logs
• Chronic-Pain Protective Behavior Detection with Deep Learning
• Learning beyond datasets: Knowledge Graph Augmented Neural Networks for Natural language Processing
• Recomposition vs. Prediction: A Novel Anomaly Detection for Discrete Events Based On Autoencoder
• Sliced Recurrent Neural Networks
• An LSTM approach to Forecast Migration using Google Trends
• A deep learning based solution for construction equipment detection: from development to deployment
• Strongly-Typed Recurrent Neural Networks
• Inception-inspired LSTM for Next-frame Video Prediction
• An Explicit Local and Global Representation Disentanglement Framework with Applications in Deep Clustering and Unsupervised Object Detection
• Neural Networks for Scalar Input and Functional Output
• Human Silhouette and Skeleton Video Synthesis through Wi-Fi signals
• Modelling Student Behavior using Granular Large Scale Action Data from a MOOC
• Deep Recurrent Neural Network for Protein Function Prediction from Sequence
• Detection and Tracking of Liquids with Fully Convolutional Networks
• Learning semantic sentence representations from visually grounded language without lexical knowledge
• The effect of phased recurrent units in the classification of multiple catalogs of astronomical lightcurves
• Deep Spatial Regression Model for Image Crowd Counting
• Efficiency Evaluation of Character-level RNN Training Schedules
• Restricted Recurrent Neural Networks
• Recurrent Off-policy Baselines for Memory-based Continuous Control
• Leveraging World Events to Predict E-Commerce Consumer Demand under Anomaly
• Ratatouille: A tool for Novel Recipe Generation
• Adaptive Lighting for Data-Driven Non-Line-of-Sight 3D Localization and Object Identification
• Improving the Performance of Neural Machine Translation Involving Morphologically Rich Languages
• An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platforms
• Probabilistic Multi-modal Trajectory Prediction with Lane Attention for Autonomous Vehicles
• Reduced-Gate Convolutional LSTM Using Predictive Coding for Spatiotemporal Prediction
• Learning Regularity in Skeleton Trajectories for Anomaly Detection in Videos
• Assessment of the Relative Importance of different hyper-parameters of LSTM for an IDS
• An Open-Source and Reproducible Implementation of LSTM and GRU Networks for Time Series Forecasting
• A Survey on Machine Learning Techniques for Source Code Analysis
• Smart City Enabled by 5G/6G Networks: An Intelligent Hybrid Random Access Scheme
• Deep Evidential Regression
• Convolutional Neural Network based Multiple-Rate Compressive Sensing for Massive MIMO CSI Feedback: Design, Simulation, and Analysis
• RNNFast: An Accelerator for Recurrent Neural Networks Using Domain Wall Memory
• Pair the Dots: Jointly Examining Training History and Test Stimuli for Model Interpretability
• Generative ensemble deep learning severe weather prediction from a deterministic convection-allowing model
• Neural Forecasting of the Italian Sovereign Bond Market with Economic News
• Autoencoder-based Condition Monitoring and Anomaly Detection Method for Rotating Machines
• MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response
• Recurrent Neural Network Encoder with Attention for Community Question Answering
• A Way out of the Odyssey: Analyzing and Combining Recent Insights for LSTMs
• Deep Learning applied to Road Traffic Speed forecasting
• Stacked Cross-modal Feature Consolidation Attention Networks for Image Captioning
• AutoCaption: Image Captioning with Neural Architecture Search
• Recurrent Neural Network Assisted Transmitter Selection for Secrecy in Cognitive Radio Network
• UniNet: Next Term Course Recommendation using Deep Learning
• Forecasting Solar Energetic Proton Integral Fluxes with Bi-Directional Long Short-Term Memory Neural Networks
• A Neural Architecture Mimicking Humans End-to-End for Natural Language Inference
• Proactive Resource Management for LTE in Unlicensed Spectrum: A Deep Learning Perspective
• Performance versus Complexity Study of Neural Network Equalizers in Coherent Optical Systems
• Learning Video Instance Segmentation with Recurrent Graph Neural Networks
• Lexicons and Minimum Risk Training for Neural Machine Translation: NAIST-CMU at WAT2016
• Hierarchical Predictive Coding Models in a Deep-Learning Framework
• Social Behavior Prediction from First Person Videos
• Towards Playlist Generation Algorithms Using RNNs Trained on Within-Track Transitions
• Recurrent Neural Networks for anomaly detection in the Post-Mortem time series of LHC superconducting magnets
• Recurrent neural network-based user authentication for freely typed keystroke data
• On the quantization of recurrent neural networks
• Context Aware Machine Learning
• Deep Spiking Neural Networks for Large Vocabulary Automatic Speech Recognition
• Model predictive control design for dynamical systems learned by Long Short-Term Memory Networks
• Interpretation of Deep Temporal Representations by Selective Visualization of Internally Activated Nodes
• A Fair Federated Learning Framework for Collaborative Network Traffic Prediction and Resource Allocation
• A Model-Driven Stack-Based Fully Convolutional Network for Pancreas Segmentation
• Deep Learning Based on Generative Adversarial and Convolutional Neural Networks for Financial Time Series Predictions
• Initializing LSTM internal states via manifold learning
• Cached Long Short-Term Memory Neural Networks for Document-Level Sentiment Classification
• DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction
• Explainable artificial intelligence for mechanics: physics-informing neural networks for constitutive models
• Voice Cloning: a Multi-Speaker Text-to-Speech Synthesis Approach based on Transfer Learning
• Object Recognition from Short Videos for Robotic Perception
• Low-rank passthrough neural networks
• A Hybrid Framework for Sequential Data Prediction with End-to-End Optimization
• Echo: Compiler-based GPU Memory Footprint Reduction for LSTM RNN Training
• Long Short-Term Attention
• Difference Attention Based Error Correction LSTM Model for Time Series Prediction
• A system of different layers of abstraction for artificial intelligence
• Time Matters: Time-Aware LSTMs for Predictive Business Process Monitoring
• HeterPS: Distributed Deep Learning With Reinforcement Learning Based Scheduling in Heterogeneous Environments
• Evolutionary Multi-objective Optimisation in Neurotrajectory Prediction
• Clustering in Recurrent Neural Networks for Micro-Segmentation using Spending Personality
• Assessing Language Models with Scaling Properties
• Attentive Weakly Supervised land cover mapping for object-based satellite image time series data with spatial interpretation
• Why ResNet Works? Residuals Generalize
• Transformers for Limit Order Books
• Referring Expression Comprehension: A Survey of Methods and Datasets
• Effectiveness and predictability of in-network storage cache for scientific workflows
• Early Prediction for Physical Human Robot Collaboration in the Operating Room
• Batch-normalized Recurrent Highway Networks
• A Combination of Temporal Sequence Learning and Data Description for Anomaly-based NIDS
• A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks
• dynoNet: a neural network architecture for learning dynamical systems
• KSR: A Semantic Representation of Knowledge Graph within a Novel Unsupervised Paradigm
• LSTM-based Anomaly Detection for Non-linear Dynamical System
• Access Trends of In-network Cache for Scientific Data
• Frame-To-Frame Consistent Semantic Segmentation
• Deep Learning based Channel Estimation Algorithm over Time Selective Fading Channels
• A semi-holographic hyperdimensional representation system for hardware-friendly cognitive computing
• An Empirical Study of Excitation and Aggregation Design Adaptions in CLIP4Clip for Video-Text Retrieval
• Learning to Disambiguate by Asking Discriminative Questions
• Deep Learning based Pedestrian Inertial Navigation: Methods, Dataset and On-Device Inference
• Using stigmergy as a computational memory in the design of recurrent neural networks
• Real-time interactive sequence generation and control with Recurrent Neural Network ensembles
• Pattern-based Long Short-term Memory for Mid-term Electrical Load Forecasting
• Joint Learning of Domain Classification and Out-of-Domain Detection with Dynamic Class Weighting for Satisficing False Acceptance Rates
• A Dynamic Model for Traffic Flow Prediction Using Improved DRN
• Communication-Efficient Agnostic Federated Averaging
• Forecasting Energy Availability in Local Energy Communities via LSTM Federated Learning
• Weakly-Supervised Temporal Action Localization Through Local-Global Background Modeling
• MC-QDSNN: Quantized Deep evolutionary SNN with Multi-Dendritic Compartment Neurons for Stress Detection using Physiological Signals
• Empirical Evaluation of RNN Architectures on Sentence Classification Task
• Product Function Need Recognition via Semi-supervised Attention Network
• Data-driven Smart Ponzi Scheme Detection
• Deep Convolutional Neural Networks for Anomaly Event Classification on Distributed Systems
• A deep neural network to enhance prediction of 1-year mortality using echocardiographic videos of the heart
• A Tree Architecture of LSTM Networks for Sequential Regression with Missing Data
• Encoding-based Memory Modules for Recurrent Neural Networks
• Precipitation Nowcasting: Leveraging bidirectional LSTM and 1D CNN
• A Model-driven and Data-driven Fusion Framework for Accurate Air Quality Prediction
• Chinese Spelling Error Detection Using a Fusion Lattice LSTM
• Refined Gate: A Simple and Effective Gating Mechanism for Recurrent Units
• A Dynamically Controlled Recurrent Neural Network for Modeling Dynamical Systems
• Short-term load forecasting using optimized LSTM networks based on EMD
• Deep learning for brake squeal: vibration detection, characterization and prediction
• Learned Indexes for Dynamic Workloads
• Embeddings and Representation Learning for Structured Data
• MuFuRU: The Multi-Function Recurrent Unit
• LSTM Hyper-Parameter Selection for Malware Detection: Interaction Effects and Hierarchical Selection Approach
• Training Recurrent Neural Networks against Noisy Computations during Inference
• Data Driven VRP: A Neural Network Model to Learn Hidden Preferences for VRP
• Predicting Maximum Permitted Process Forces for Object Grasping and Manipulation Using a Deep Learning Regression Model
• WSRNet: Joint Spotting and Recognition of Handwritten Words
• Dependency Parsing with LSTMs: An Empirical Evaluation
• ViPTT-Net: Video pretraining of spatio-temporal model for tuberculosis type classification from chest CT scans
• In folly ripe. In reason rotten. Putting machine theology to rest
• Dual-side Sparse Tensor Core
• Recurrent neural network training with preconditioned stochastic gradient descent
• Object-oriented Neural Programming (OONP) for Document Understanding
• Open-Ended Multi-Modal Relational Reasoning for Video Question Answering
• Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Time
• Predicting the Ocean Currents using Deep Learning
• An LSTM-Aided Hybrid Random Access Scheme for 6G Machine Type Communication Networks
• MoNetV2: Enhanced Motion Network for Freehand 3D Ultrasound Reconstruction
• Memory and attention in deep learning
• Advanced control based on Recurrent Neural Networks learned using Virtual Reference Feedback Tuning and application to an Electronic Throttle Body (with supplementary material)
• Structured Convolutional Kernel Networks for Airline Crew Scheduling
• A Multivariate Density Forecast Approach for Online Power System Security Assessment
• Bi-directional LSTM Recurrent Neural Network for Chinese Word Segmentation
• Metaparametric Neural Networks for Survival Analysis
• Segmentation and Optimal Region Selection of Physiological Signals using Deep Neural Networks and Combinatorial Optimization
• RNN-based Online Learning: An Efficient First-Order Optimization Algorithm with a Convergence Guarantee
• Deep Learning Calabi-Yau four folds with hybrid and recurrent neural network architectures
• Improving LSTM Neural Networks for Better Short-Term Wind Power Predictions
• Sequential Embedding-based Attentive (SEA) classifier for malware classification
• Tweets Can Tell: Activity Recognition using Hybrid Long Short-Term Memory Model
• Accelerator-Aware Training for Transducer-Based Speech Recognition
• Deep Learning Detection of Inaccurate Smart Electricity Meters: A Case Study
• Borrowing from Similar Code: A Deep Learning NLP-Based Approach for Log Statement Automation
• PowerNet: Neural Power Demand Forecasting in Smart Grid
• Recurrent Neural Network from Adder's Perspective: Carry-lookahead RNN
• The Image Local Autoregressive Transformer
• Augmented Shortcuts for Vision Transformers
• Asset Price Forecasting using Recurrent Neural Networks
• Deep Learning Scooping Motion using Bilateral Teleoperations
• A Neural Network-Based Linguistic Similarity Measure for Entrainment in Conversations
• StressNet: Detecting Stress in Thermal Videos
• Recurrent Neural Networks with Flexible Gates using Kernel Activation Functions
• Influence Paths for Characterizing Subject-Verb Number Agreement in LSTM Language Models
• Sampling-free Uncertainty Estimation in Gated Recurrent Units with Exponential Families
• Automatic Dialogic Instruction Detection for K-12 Online One-on-one Classes
• Sensor Fusion: Gated Recurrent Fusion to Learn Driving Behavior from Temporal Multimodal Data
• Language Transfer of Audio Word2Vec: Learning Audio Segment Representations without Target Language Data
• Prototypical Recurrent Unit
• Optimizing Recurrent Neural Networks Architectures under Time Constraints
• You Only Look & Listen Once: Towards Fast and Accurate Visual Grounding
• Statistical Parametric Speech Synthesis Using Bottleneck Representation From Sequence Auto-encoder
• Multi-Task Learning for Argumentation Mining
• De-identification of medical records using conditional random fields and long short-term memory networks
• A Conceptual Development of Quench Prediction App build on LSTM and ELQA framework
• Empirical Evaluation of A New Approach to Simplifying Long Short-term Memory (LSTM)
• Teaching a Machine to Diagnose a Heart Disease; Beginning from digitizing scanned ECGs to detecting the Brugada Syndrome (BrS)
• Price Optimization in Fashion E-commerce
• PDRNN: Modular Data-driven Pedestrian Dead Reckoning on Loosely Coupled Radio- and Inertial-Signalstreams
• Forecasting Empty Container availability for Vehicle Booking System Application
• AMI-FML: A Privacy-Preserving Federated Machine Learning Framework for AMI
• YNU-HPCC at SemEval-2020 Task 8: Using a Parallel-Channel Model for Memotion Analysis
• Deep Learning-based Stress Determinator for Mouse Psychiatric Analysis using Hippocampus Activity
• Challenges and Thrills of Legal Arguments
• Neural Network Architecture for Credibility Assessment of Textual Claims
• DDLSTM: Dual-Domain LSTM for Cross-Dataset Action Recognition
• CASU2Net: Cascaded Unification Network by a Two-step Early Fusion for Fault Detection in Offshore Wind Turbines
• Layer Dynamics of Linearised Neural Nets
• Estimating Forces of Robotic Pouring Using a LSTM RNN
• Artificial intelligence prediction of stock prices using social media
• Learning Longer-term Dependencies via Grouped Distributor Unit
• Dynamical prediction of two meteorological factors using the deep neural network and the long short term memory $(1)$
• Investigation of the Impacts of COVID-19 on the Electricity Consumption of a University Dormitory Using Weather Normalization
• Rapid Autonomous Car Control based on Spatial and Temporal Visual Cues
• Long Short Term Memory Networks for Bandwidth Forecasting in Mobile Broadband Networks under Mobility
• Bayesian Optimization and Deep Learning forsteering wheel angle prediction
• Improving the Thermal Infrared Monitoring of Volcanoes: A Deep Learning Approach for Intermittent Image Series
• XJTLUIndoorLoc: A New Fingerprinting Database for Indoor Localization and Trajectory Estimation Based on Wi-Fi RSS and Geomagnetic Field
• Counting in Language with RNNs
• RotLSTM: Rotating Memories in Recurrent Neural Networks
• MIMO Channel Information Feedback Using Deep Recurrent Network
• Sequential Variational Autoencoders for Collaborative Filtering
• Improving Chinese SRL with Heterogeneous Annotations
• RL-NCS: Reinforcement learning based data-driven approach for nonuniform compressed sensing
• Dual Attention Network for Product Compatibility and Function Satisfiability Analysis
• Dual-modality seq2seq network for audio-visual event localization
• Non-Negative PARATUCK2 Tensor Decomposition Combined to LSTM Network For Smart Contracts Profiling
• Time-domain deep learning filtering of structured atmospheric noise for ground-based millimeter astronomy
• Water Quality Prediction on a Sigfox-compliant IoT Device: The Road Ahead of WaterS
• Temporally Folded Convolutional Neural Networks for Sequence Forecasting
• Perceiving and Reasoning About Liquids Using Fully Convolutional Networks
• Supervised Domain Enablement Attention for Personalized Domain Classification
• Efficient Global String Kernel with Random Features: Beyond Counting Substructures
• Bit-level Optimized Neural Network for Multi-antenna Channel Quantization
• Multiple Generative Models Ensemble for Knowledge-Driven Proactive Human-Computer Dialogue Agent
• Effect of Various Regularizers on Model Complexities of Neural Networks in Presence of Input Noise
• How LSTM Encodes Syntax: Exploring Context Vectors and Semi-Quantization on Natural Text