Publications (20)
Cost-aware Bayesian Optimization
Eric Hans Lee, Valerio Perrone, Cedric Archambeau +1
Bayesian optimization (BO) is a class of global optimization algorithms, suitable for minimizing an expensive objective function in as few function evaluations as possible. While B…
Constrained Bayesian Optimization with Max-Value Entropy Search
Valerio Perrone, Iaroslav Shcherbatyi, Rodolphe Jenatton +2
Bayesian optimization (BO) is a model-based approach to sequentially optimize expensive black-box functions, such as the validation error of a deep neural network with respect to i…
Amazon SageMaker Autopilot: a white box AutoML solution at scale
Piali Das, Valerio Perrone, Nikita Ivkin +22
AutoML systems provide a black-box solution to machine learning problems by selecting the right way of processing features, choosing an algorithm and tuning the hyperparameters of…
A Nonmyopic Approach to Cost-Constrained Bayesian Optimization
Eric Hans Lee, David Eriksson, Valerio Perrone +1
Bayesian optimization (BO) is a popular method for optimizing expensive-to-evaluate black-box functions. BO budgets are typically given in iterations, which implicitly assumes each…
Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning
Valerio Perrone, Huibin Shen, Matthias Seeger +2
Bayesian optimization (BO) is a successful methodology to optimize black-box functions that are expensive to evaluate. While traditional methods optimize each black-box function in…
A Quantile-based Approach for Hyperparameter Transfer Learning
David Salinas, Huibin Shen, Valerio Perrone
Bayesian optimization (BO) is a popular methodology to tune the hyperparameters of expensive black-box functions. Traditionally, BO focuses on a single task at a time and is not de…
Amazon SageMaker Automatic Model Tuning: Scalable Gradient-Free Optimization
Valerio Perrone, Huibin Shen, Aida Zolic +12
Tuning complex machine learning systems is challenging. Machine learning typically requires to set hyperparameters, be it regularization, architecture, or optimization parameters,…
Poisson Random Fields for Dynamic Feature Models
Valerio Perrone, Paul A. Jenkins, Dario Spano +1
We present the Wright-Fisher Indian buffet process (WF-IBP), a probabilistic model for time-dependent data assumed to have been generated by an unknown number of latent features. T…
Hyperparameter Optimization in Machine Learning
Luca Franceschi, Michele Donini, Valerio Perrone +5
Hyperparameters are configuration variables controlling the behavior of machine learning algorithms. They are ubiquitous in machine learning and artificial intelligence and the cho…
Automatic Termination for Hyperparameter Optimization
Anastasia Makarova, Huibin Shen, Valerio Perrone +5
Bayesian optimization (BO) is a widely popular approach for the hyperparameter optimization (HPO) in machine learning. At its core, BO iteratively evaluates promising configuration…
A Simple and Fast Baseline for Tuning Large XGBoost Models
Sanyam Kapoor, Valerio Perrone
XGBoost, a scalable tree boosting algorithm, has proven effective for many prediction tasks of practical interest, especially using tabular datasets. Hyperparameter tuning can furt…
A multi-objective perspective on jointly tuning hardware and hyperparameters
David Salinas, Valerio Perrone, Olivier Cruchant +1
In addition to the best model architecture and hyperparameters, a full AutoML solution requires selecting appropriate hardware automatically. This can be framed as a multi-objectiv…
A Likelihood-Free Inference Framework for Population Genetic Data using Exchangeable Neural Networks
Jeffrey Chan, Valerio Perrone, Jeffrey P. Spence +3
An explosion of high-throughput DNA sequencing in the past decade has led to a surge of interest in population-scale inference with whole-genome data. Recent work in population gen…
Lexical semantic change for Ancient Greek and Latin
Valerio Perrone, Simon Hengchen, Marco Palma +3
Change and its precondition, variation, are inherent in languages. Over time, new words enter the lexicon, others become obsolete, and existing words acquire new senses. Associatin…
Multiple Adaptive Bayesian Linear Regression for Scalable Bayesian Optimization with Warm Start
Valerio Perrone, Rodolphe Jenatton, Matthias Seeger +1
Bayesian optimization (BO) is a model-based approach for gradient-free black-box function optimization. Typically, BO is powered by a Gaussian process (GP), whose algorithmic compl…
Structural Pruning of Pre-trained Language Models via Neural Architecture Search
Aaron Klein, Jacek Golebiowski, Xingchen Ma +2
Pre-trained language models (PLM), for example BERT or RoBERTa, mark the state-of-the-art for natural language understanding task when fine-tuned on labeled data. However, their la…
GASC: Genre-Aware Semantic Change for Ancient Greek
Valerio Perrone, Marco Palma, Simon Hengchen +3
Word meaning changes over time, depending on linguistic and extra-linguistic factors. Associating a word's correct meaning in its historical context is a central challenge in diach…
Pareto-efficient Acquisition Functions for Cost-Aware Bayesian Optimization
Gauthier Guinet, Valerio Perrone, Cédric Archambeau
Bayesian optimization (BO) is a popular method to optimize expensive black-box functions. It efficiently tunes machine learning algorithms under the implicit assumption that hyperp…
Relativistic Monte Carlo
Xiaoyu Lu, Valerio Perrone, Leonard Hasenclever +2
Hamiltonian Monte Carlo (HMC) is a popular Markov chain Monte Carlo (MCMC) algorithm that generates proposals for a Metropolis-Hastings algorithm by simulating the dynamics of a Ha…
Fair Bayesian Optimization
Valerio Perrone, Michele Donini, Muhammad Bilal Zafar +3
Given the increasing importance of machine learning (ML) in our lives, several algorithmic fairness techniques have been proposed to mitigate biases in the outcomes of the ML model…