Publications (15)
Latent Bottlenecked Attentive Neural Processes
Leo Feng, Hossein Hajimirsadeghi, Yoshua Bengio +1
Neural Processes (NPs) are popular methods in meta-learning that can estimate predictive uncertainty on target datapoints by conditioning on a context dataset. Previous state-of-th…
Towards Better Selective Classification
Leo Feng, Mohamed Osama Ahmed, Hossein Hajimirsadeghi +1
We tackle the problem of Selective Classification where the objective is to achieve the best performance on a predetermined ratio (coverage) of the dataset. Recent state-of-the-art…
Exploration in Approximate Hyper-State Space for Meta Reinforcement Learning
Luisa Zintgraf, Leo Feng, Cong Lu +4
To rapidly learn a new task, it is often essential for agents to explore efficiently -- especially when performance matters from the first timestep. One way to learn such behaviour…
Continuous-Time Meta-Learning with Forward Mode Differentiation
Tristan Deleu, David Kanaa, Leo Feng +4
Drawing inspiration from gradient-based meta-learning methods with infinitely small gradient steps, we introduce Continuous-Time Meta-Learning (COMLN), a meta-learning algorithm wh…
Were RNNs All We Needed?
Leo Feng, Frederick Tung, Mohamed Osama Ahmed +2
The introduction of Transformers in 2017 reshaped the landscape of deep learning. Originally proposed for sequence modelling, Transformers have since achieved widespread success ac…
Adaptive teachers for amortized samplers
Minsu Kim, Sanghyeok Choi, Taeyoung Yun +7
Amortized inference is the task of training a parametric model, such as a neural network, to approximate a distribution with a given unnormalized density where exact sampling is in…
VIABLE: Fast Adaptation via Backpropagating Learned Loss
Leo Feng, Luisa Zintgraf, Bei Peng +1
In few-shot learning, typically, the loss function which is applied at test time is the one we are ultimately interested in minimising, such as the mean-squared-error loss for a re…
Memory Efficient Neural Processes via Constant Memory Attention Block
Leo Feng, Frederick Tung, Hossein Hajimirsadeghi +2
Neural Processes (NPs) are popular meta-learning methods for efficiently modelling predictive uncertainty. Recent state-of-the-art methods, however, leverage expensive attention me…
Can LLMs Take Retrieved Information with a Grain of Salt?
Behzad Shayegh, Mohamed Osama Ahmed, Fred Tung +1
Large language models have demonstrated impressive retrieval-augmented capabilities. However, a crucial area remains underexplored: their ability to appropriately adapt responses t…
Generative Active Learning for the Search of Small-molecule Protein Binders
Maksym Korablyov, Cheng-Hao Liu, Moksh Jain +31
Despite substantial progress in machine learning for scientific discovery in recent years, truly de novo design of small molecules which exhibit a property of interest remains a si…
Attention as an RNN
Leo Feng, Frederick Tung, Hossein Hajimirsadeghi +3
The advent of Transformers marked a significant breakthrough in sequence modelling, providing a highly performant architecture capable of leveraging GPU parallelism. However, Trans…
Designing Biological Sequences via Meta-Reinforcement Learning and Bayesian Optimization
Leo Feng, Padideh Nouri, Aneri Muni +2
The ability to accelerate the design of biological sequences can have a substantial impact on the progress of the medical field. The problem can be framed as a global optimization…
Do LLMs Benefit from User and Item Embeddings in Recommendation Tasks?
Mir Rayat Imtiaz Hossain, Leo Feng, Leonid Sigal +1
Large Language Models (LLMs) have emerged as promising recommendation systems, offering novel ways to model user preferences through generative approaches. However, many existing m…
Constant Memory Attention Block
Leo Feng, Frederick Tung, Hossein Hajimirsadeghi +2
Modern foundation model architectures rely on attention mechanisms to effectively capture context. However, these methods require linear or quadratic memory in terms of the number…
Tree Cross Attention
Leo Feng, Frederick Tung, Hossein Hajimirsadeghi +2
Cross Attention is a popular method for retrieving information from a set of context tokens for making predictions. At inference time, for each prediction, Cross Attention scans th…