Automated Architecture Design for Deep Neural Networks

Abstract

Machine learning has made tremendous progress in recent years and received large amounts of public attention. Though we are still far from designing a full artificially intelligent agent, machine learning has brought us many applications in which computers solve human learning tasks remarkably well. Much of this progress comes from a recent trend within machine learning, called deep learning. Deep learning models are responsible for many state-of-the-art applications of machine learning. Despite their success, deep learning models are hard to train, very difficult to understand, and often times so complex that training is only possible on very large GPU clusters. Lots of work has been done on enabling neural networks to learn efficiently. However, the design and architecture of such neural networks is often done manually through trial and error and expert knowledge. This thesis inspects different approaches, existing and novel, to automate the design of deep feedforward neural networks in an attempt to create less complex models with good performance that take away the burden of deciding on an architecture and make it more efficient to design and train such deep networks.

arXiv commentUndergraduate Thesis

References in corpus (9)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• An overview of gradient descent optimization algorithms
• Neural Architecture Search: A Survey
• OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
• Understanding deep learning requires rethinking generalization
• The Loss Surfaces of Multilayer Networks
• Exponential expressivity in deep neural networks through transient chaos
• Representation Benefits of Deep Feedforward Networks
• Forward Thinking: Building and Training Neural Networks One Layer at a Time

Cited by in corpus (4)

• On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice
• IoT Data Analytics in Dynamic Environments: From An Automated Machine Learning Perspective
• Total Nitrogen Estimation in Agricultural Soils via Aerial Multispectral Imaging and LIBS
• Search Algorithms for Automated Hyper-Parameter Tuning