19 citations · 41 across the 10 of their papers we have counts for
11 papers
T-RoPE: Time-Aware Rotary Position Embedding for Sequential Recommendation
Yang Liu, Noel Loo, Ali Khanafer +4
Large-scale recommenders increasingly adopt the sequential generative recipe behind large language models, bringing the Transformer into recommendation along with design choices ma…
Preconditioned DeltaNet: Curvature-aware Sequence Modeling for Linear Recurrences
Neehal Tumma, Noel Loo, Daniela Rus
To address the increasing long-context compute limitations of softmax attention, several subquadratic recurrent operators have been developed. This work includes models such as Mam…
LFM2 Technical Report
Alexander Amini, Anna Banaszak, Harold Benoit +30
We present LFM2, a family of Liquid Foundation Models designed for efficient on-device deployment and strong task capabilities. Using hardware-in-the-loop architecture search under…
Linear Projections of Teacher Embeddings for Few-Class Distillation
Noel Loo, Fotis Iliopoulos, Wei Hu +1
Knowledge Distillation (KD) has emerged as a promising approach for transferring knowledge from a larger, more complex teacher model to a smaller student model. Traditionally, KD i…
Leveraging Low-Rank and Sparse Recurrent Connectivity for Robust Closed-Loop Control
Neehal Tumma, Mathias Lechner, Noel Loo +2
Developing autonomous agents that can interact with changing environments is an open challenge in machine learning. Robustness is particularly important in these settings as agents…
On the Size and Approximation Error of Distilled Sets
Alaa Maalouf, Murad Tukan, Noel Loo +3
Dataset Distillation is the task of synthesizing small datasets from large ones while still retaining comparable predictive accuracy to the original uncompressed dataset. Despite s…
Dataset Distillation with Convexified Implicit Gradients
Noel Loo, Ramin Hasani, Mathias Lechner +1
We propose a new dataset distillation algorithm using reparameterization and convexification of implicit gradients (RCIG), that substantially improves the state-of-the-art. To this…
Understanding Reconstruction Attacks with the Neural Tangent Kernel and Dataset Distillation
Noel Loo, Ramin Hasani, Mathias Lechner +2
Modern deep learning requires large volumes of data, which could contain sensitive or private information that cannot be leaked. Recent work has shown for homogeneous neural networ…
Efficient Dataset Distillation Using Random Feature Approximation
Noel Loo, Ramin Hasani, Alexander Amini +1
Dataset distillation compresses large datasets into smaller synthetic coresets which retain performance with the aim of reducing the storage and computational burden of processing…
Evolution of Neural Tangent Kernels under Benign and Adversarial Training
Noel Loo, Ramin Hasani, Alexander Amini +1
Two key challenges facing modern deep learning are mitigating deep networks' vulnerability to adversarial attacks and understanding deep learning's generalization capabilities. Tow…
Generalized Variational Continual Learning
Noel Loo, Siddharth Swaroop, Richard E. Turner
Continual learning deals with training models on new tasks and datasets in an online fashion. One strand of research has used probabilistic regularization for continual learning, w…