A primer on model-guided exploration of fitness landscapes for biological sequence design

Abstract

Machine learning methods are increasingly employed to address challenges faced by biologists. One area that will greatly benefit from this cross-pollination is the problem of biological sequence design, which has massive potential for therapeutic applications. However, significant inefficiencies remain in communication between these fields which result in biologists finding the progress in machine learning inaccessible, and hinder machine learning scientists from contributing to impactful problems in bioengineering. Sequence design can be seen as a search process on a discrete, high-dimensional space, where each sequence is associated with a function. This sequence-to-function map is known as a "Fitness Landscape". Designing a sequence with a particular function is hence a matter of "discovering" such a (often rare) sequence within this space. Today we can build predictive models with good interpolation ability due to impressive progress in the synthesis and testing of biological sequences in large numbers, which enables model training and validation. However, it often remains a challenge to find useful sequences with the properties that we like using these models. In particular, in this primer we highlight that algorithms for experimental design, what we call "exploration strategies", are a related, yet distinct problem from building good models of sequence-to-function maps. We review advances and insights from current literature -- by no means a complete treatment -- while highlighting desirable features of optimal model-guided exploration, and cover potential pitfalls drawn from our own experience. This primer can serve as a starting point for researchers from different domains that are interested in the problem of searching a sequence space with a model, but are perhaps unaware of approaches that originate outside their field.

References in corpus (13)

• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
• Improved contact prediction in proteins: Using pseudolikelihoods to infer Potts models
• Improving Online Algorithms via ML Predictions
• Generating and designing DNA with deep generative models
• Inferring fitness landscapes by regression produces biased estimates of epistasis
• Evaluating Protein Transfer Learning with TAPE
• Variational auto-encoding of protein sequences
• Bayesian Optimization for Synthetic Gene Design
• AdaLead: A simple and robust adaptive greedy search algorithm for sequence design
• ProGen: Language Modeling for Protein Generation
• Model Inversion Networks for Model-Based Optimization
• Autofocused oracles for model-based design
• Batched Stochastic Bayesian Optimization via Combinatorial Constraints Design