Theano: new features and speed improvements

Abstract

Theano is a linear algebra compiler that optimizes a user's symbolically-specified mathematical computations to produce efficient low-level implementations. In this paper, we present new features and efficiency improvements to Theano, and benchmarks demonstrating Theano's performance relative to Torch7, a recently introduced machine learning library, and to RNNLM, a C++ library targeted at recurrent neural networks.

Presented at the Deep Learning Workshop, NIPS 2012

Cited by in corpus (85)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Explaining and Harnessing Adversarial Examples
• On the difficulty of training Recurrent Neural Networks
• FitNets: Hints for Thin Deep Nets
• Attention-Based Models for Speech Recognition
• Rotation-invariant convolutional neural networks for galaxy morphology prediction
• On Using Monolingual Corpora in Neural Machine Translation
• Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
• Gated Feedback Recurrent Neural Networks
• Gradient-based Hyperparameter Optimization through Reversible Learning
• Variational Autoencoder for Deep Learning of Images, Labels and Captions
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
• GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
• MADE: Masked Autoencoder for Distribution Estimation
• Discourse-Aware Rumour Stance Classification in Social Media Using Sequential Classifiers
• Blocks and Fuel: Frameworks for deep learning
• Adversarial Feature Matching for Text Generation
• A Neural Knowledge Language Model
• Protein Secondary Structure Prediction with Long Short Term Memory Networks
• Deep learning for galaxy surface brightness profile fitting
• Feature Representation in Convolutional Neural Networks
• Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks
• On the saddle point problem for non-convex optimization
• Deep Reconstruction-Classification Networks for Unsupervised Domain Adaptation
• Benchmark of Deep Learning Models on Large Healthcare MIMIC Datasets
• Tensor-Train Recurrent Neural Networks for Video Classification
• rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorch
• Visual Causal Feature Learning
• High-Resolution Shape Completion Using Deep Neural Networks for Global Structure and Local Geometry Inference
• Alzheimer's Disease Diagnostics by a Deeply Supervised Adaptable 3D Convolutional Network
• EmoNets: Multimodal deep learning approaches for emotion recognition in video
• Embedding Word Similarity with Neural Machine Translation
• Theano-based Large-Scale Visual Recognition with Multiple GPUs
• Explorations on high dimensional landscapes
• Recurrent Neural Networks with External Memory for Language Understanding
• Sales Forecast in E-commerce using Convolutional Neural Network
• From phonemes to images: levels of representation in a recurrent neural model of visually-grounded language learning
• Iterative Neural Autoregressive Distribution Estimator (NADE-k)
• Deep Directed Generative Autoencoders
• Neural Networks Designing Neural Networks: Multi-Objective Hyper-Parameter Optimization
• End-to-end Convolutional Network for Saliency Prediction
• DeepRT: deep learning for peptide retention time prediction in proteomics
• Machine learning for metal additive manufacturing: Predicting temperature and melt pool fluid dynamics using physics-informed neural networks
• Functional connectivity patterns of autism spectrum disorder identified by deep feature learning
• Large Scale Evolution of Convolutional Neural Networks Using Volunteer Computing
• Variational Inference with Hamiltonian Monte Carlo
• Neural Networks with Smooth Adaptive Activation Functions for Regression
• The Game Imitation: Deep Supervised Convolutional Networks for Quick Video Game AI
• Grid Loss: Detecting Occluded Faces
• Enabling My Robot To Play Pictionary : Recurrent Neural Networks For Sketch Recognition
• Exploiting long-term temporal dynamics for video captioning
• Vector representations of text data in deep learning
• Shape Completion Enabled Robotic Grasping
• Likelihood Estimation for Generative Adversarial Networks
• Predictive Clinical Decision Support System with RNN Encoding and Tensor Decoding
• Recurrent Neural Network based Part-of-Speech Tagger for Code-Mixed Social Media Text
• Learning language through pictures
• Tied Hidden Factors in Neural Networks for End-to-End Speaker Recognition
• MNN: A Universal and Efficient Inference Engine
• Attend in groups: a weakly-supervised deep learning framework for learning from web data
• Prediction of Prokaryotic and Eukaryotic Promoters Using Convolutional Deep Learning Neural Networks
• Satirical News Detection and Analysis using Attention Mechanism and Linguistic Features
• Action Classification and Highlighting in Videos
• Deep Learning in the Automotive Industry: Recent Advances and Application Examples
• Analytic Insights into Structure and Rank of Neural Network Hessian Maps
• A Hierarchical Recurrent Encoder-Decoder For Generative Context-Aware Query Suggestion
• Neural Network Regularization via Robust Weight Factorization
• Can deep learning help you find the perfect match?
• On the Equivalence Between Deep NADE and Generative Stochastic Networks
• Reader-Aware Multi-Document Summarization: An Enhanced Model and The First Dataset
• Max-Margin Deep Generative Models for (Semi-)Supervised Learning
• Deep Triphone Embedding Improves Phoneme Recognition
• MLitB: Machine Learning in the Browser
• Neural Variational Inference and Learning in Undirected Graphical Models
• Additive manufacturing process design with differentiable simulations
• What does fault tolerant Deep Learning need from MPI?
• Character-level Deep Conflation for Business Data Analytics
• PyTorchPipe: a framework for rapid prototyping of pipelines combining language and vision
• PZnet: Efficient 3D ConvNet Inference on Manycore CPUs
• Discriminative Enhancement for Single Channel Audio Source Separation using Deep Neural Networks
• Language discrimination and clustering via a neural network approach
• Accurate computational evolution of proteins and its dependence on deep learning
• Stacked Denoising Autoencoders and Transfer Learning for Immunogold Particles Detection and Recognition
• Optimising The Input Window Alignment in CD-DNN Based Phoneme Recognition for Low Latency Processing
• Understanding Minimum Probability Flow for RBMs Under Various Kinds of Dynamics