A Genetic Algorithm with Tree-structured Mutation for Hyperparameter Optimisation of Graph Neural Networks
arXiv:2102.11995
Abstract
In recent years, graph neural networks (GNNs) have gained increasing attention, as they possess the excellent capability of processing graph-related problems. In practice, hyperparameter optimisation (HPO) is critical for GNNs to achieve satisfactory results, but this process is costly because the evaluations of different hyperparameter settings require excessively training many GNNs. Many approaches have been proposed for HPO, which aims to identify promising hyperparameters efficiently. In particular, the genetic algorithm (GA) for HPO has been explored, which treats GNNs as a black-box model, of which only the outputs can be observed given a set of hyperparameters. However, because GNN models are sophisticated and the evaluations of hyperparameters on GNNs are expensive, GA requires advanced techniques to balance the exploration and exploitation of the search and make the optimisation more effective given limited computational resources. Therefore, we proposed a tree-structured mutation strategy for GA to alleviate this issue. Meanwhile, we reviewed the recent HPO works, which gives room for the idea of tree-structure to develop, and we hope our approach can further improve these HPO methods in the future.
References in corpus (8)
- Practical Bayesian Optimization of Machine Learning Algorithms
- Graph Neural Networks: A Review of Methods and Applications
- Convolutional Networks on Graphs for Learning Molecular Fingerprints
- BayesOpt: A Bayesian Optimization Library for Nonlinear Optimization, Experimental Design and Bandits
- CMA-ES for Hyperparameter Optimization of Deep Neural Networks
- Automated Machine Learning: State-of-The-Art and Open Challenges
- Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm
- A Novel Genetic Algorithm with Hierarchical Evaluation Strategy for Hyperparameter Optimisation of Graph Neural Networks