Tetra-AML: Automatic Machine Learning via Tensor Networks

Abstract

Neural networks have revolutionized many aspects of society but in the era of huge models with billions of parameters, optimizing and deploying them for commercial applications can require significant computational and financial resources. To address these challenges, we introduce the Tetra-AML toolbox, which automates neural architecture search and hyperparameter optimization via a custom-developed black-box Tensor train Optimization algorithm, TetraOpt. The toolbox also provides model compression through quantization and pruning, augmented by compression using tensor networks. Here, we analyze a unified benchmark for optimizing neural networks in computer vision tasks and show the superior performance of our approach compared to Bayesian optimization on the CIFAR-10 dataset. We also demonstrate the compression of ResNet-18 neural networks, where we use 14.5 times less memory while losing just 3.2% of accuracy. The presented framework is generic, not limited by computer vision problems, supports hardware acceleration (such as with GPUs and TPUs) and can be further extended to quantum hardware and to hybrid quantum machine learning models.

References in corpus (6)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Neural Architecture Search: A Survey
• A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers
• NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size
• Hybrid quantum ResNet for car classification and its hyperparameter optimization
• Compressing Recurrent Neural Networks with Tensor Ring for Action Recognition