495 citations · 521 across the 8 of their papers we have counts for
4 papers · 1 filter
Not All Ranks Are Equal: Budget-Aware LoRA Merging Across Tasks
Avinash Amballa, Yashas Malur Saidutta, Wenbo Li +2
Merging low-rank adapters (LoRAs) promises to eliminate the overhead of swapping task-specific weights at inference time. However, existing merging methods assume every layer needs…
Adaptive Memory Replay for Continual Learning
James Seale Smith, Lazar Valkov, Shaunak Halbe +4
Foundation Models (FMs) have become the hallmark of modern AI, however, these models are trained on massive data, leading to financially expensive training. Updating FMs as new dat…
A Probabilistic Framework for Modular Continual Learning
Lazar Valkov, Akash Srivastava, Swarat Chaudhuri +1
Modular approaches that use a different composition of modules for each problem are a promising direction in continual learning (CL). However, searching through the large, discrete…
HOUDINI: Lifelong Learning as Program Synthesis
Lazar Valkov, Dipak Chaudhari, Akash Srivastava +2
We present a neurosymbolic framework for the lifelong learning of algorithmic tasks that mix perception and procedural reasoning. Reusing high-level concepts across domains and lea…