75 citations · 145 across the 12 of their papers we have counts for
10 papers · 1 filter
Constant Memory Attention Block
Leo Feng, Frederick Tung, Hossein Hajimirsadeghi +2
Modern foundation model architectures rely on attention mechanisms to effectively capture context. However, these methods require linear or quadratic memory in terms of the number…
Memory Efficient Neural Processes via Constant Memory Attention Block
Leo Feng, Frederick Tung, Hossein Hajimirsadeghi +2
Neural Processes (NPs) are popular meta-learning methods for efficiently modelling predictive uncertainty. Recent state-of-the-art methods, however, leverage expensive attention me…
Meta Temporal Point Processes
Wonho Bae, Mohamed Osama Ahmed, Frederick Tung +1
A temporal point process (TPP) is a stochastic process where its realization is a sequence of discrete events in time. Recent work in TPPs model the process using a neural network…
Gumbel-Softmax Selective Networks
Mahmoud Salem, Mohamed Osama Ahmed, Frederick Tung +1
ML models often operate within the context of a larger system that can adapt its response when the ML model is uncertain, such as falling back on safe defaults or a human in the lo…
Latent Bottlenecked Attentive Neural Processes
Leo Feng, Hossein Hajimirsadeghi, Yoshua Bengio +1
Neural Processes (NPs) are popular methods in meta-learning that can estimate predictive uncertainty on target datapoints by conditioning on a context dataset. Previous state-of-th…
Towards Better Selective Classification
Leo Feng, Mohamed Osama Ahmed, Hossein Hajimirsadeghi +1
We tackle the problem of Selective Classification where the objective is to achieve the best performance on a predetermined ratio (coverage) of the dataset. Recent state-of-the-art…
Monotonicity Regularization: Improved Penalties and Novel Applications to Disentangled Representation Learning and Robust Classification
Joao Monteiro, Mohamed Osama Ahmed, Hossein Hajimirsadeghi +1
We study settings where gradient penalties are used alongside risk minimization with the goal of obtaining predictors satisfying different notions of monotonicity. Specifically, we…
Point Process Flows
Nazanin Mehrasa, Ruizhi Deng, Mohamed Osama Ahmed +5
Event sequences can be modeled by temporal point processes (TPPs) to capture their asynchronous and probabilistic nature. We propose an intensity-free framework that directly model…
Combining Bayesian Optimization and Lipschitz Optimization
Mohamed Osama Ahmed, Sharan Vaswani, Mark Schmidt
Bayesian optimization and Lipschitz optimization have developed alternative techniques for optimizing black-box functions. They each exploit a different form of prior about the fun…
Stop Wasting My Gradients: Practical SVRG
Reza Babanezhad, Mohamed Osama Ahmed, Alim Virani +3
We present and analyze several strategies for improving the performance of stochastic variance-reduced gradient (SVRG) methods. We first show that the convergence rate of these met…