MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning

Abstract

Efficient approximation lies at the heart of large-scale machine learning problems. In this paper, we propose a novel, robust maximum entropy algorithm, which is capable of dealing with hundreds of moments and allows for computationally efficient approximations. We showcase the usefulness of the proposed method, its equivalence to constrained Bayesian variational inference and demonstrate its superiority over existing approaches in two applications, namely, fast log determinant estimation and information-theoretic Bayesian optimisation.

18 pages, 3 figures, Published at Entropy 2019: Special Issue Entropy Based Inference and Optimization in Machine Learning

References in corpus (8)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Determinantal point processes for machine learning
• Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
• Probabilistic Numerics and Uncertainty in Computations
• Maximum entropy and the problem of moments: A stable algorithm
• Large-scale Log-determinant Computation through Stochastic Chebyshev Expansions
• Scalable Log Determinants for Gaussian Process Kernel Learning
• Application of the Maximum relative Entropy method to the physics of ferromagnetic materials