Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Lifelong In-Context Learning with Transformers Requires Parametric Forms of Attention
Luke McDermott, Robert W. Heath, Rahul Parhi
Lifelong continual learning remains an obstacle on the path to human-like intelligence. Modern transformers show sparks of intelligence with in-context learning. The quadratic natu…
cs.LG2025
Finding Stable Subnetworks at Initialization with Dataset Distillation
Luke McDermott, Rahul Parhi
Recent works have shown that Dataset Distillation, the process for summarizing the training data, can be leveraged to accelerate the training of deep learning models. However, its…