5 papers
JumpLoRA: Sparse Adapters for Continual Learning in Large Language Models
Alexandra Dragomir, Ioana Pintilie, Antonio Barbalau +6
Adapter-based methods have become a cost-effective approach to continual learning (CL) for Large Language Models (LLMs), by sequentially learning a low-rank update matrix for each…
Temporal Taskification in Streaming Continual Learning: A Source of Evaluation Instability
Nicolae Filat, Ahmed Hussain, Konstantinos Kalogiannis +1
Streaming Continual Learning (CL) typically converts a continuous stream into a sequence of discrete tasks through temporal partitioning. We argue that this temporal taskification…
Closing the gap on tabular data with Fourier and Implicit Categorical Features
Marius Dragoi, Florin Gogianu, Elena Burceanu
While Deep Learning has demonstrated impressive results in applications on various data types, it continues to lag behind tree-based methods when applied to tabular data, often ref…
Rethinking Sparse Autoencoders: Select-and-Project for Fairness and Control from Encoder Features Alone
Antonio Bărbălau, Cristian Daniel Păduraru, Teodor Poncu +2
Sparse Autoencoders (SAEs) are widely employed for mechanistic interpretability and model steering. Within this context, steering is by design performed by means of decoding altere…
Not All Splits Are Equal: Rethinking Attribute Generalization Across Unrelated Categories
Liviu Nicolae Fircă, Antonio Bărbălau, Dan Oneata +1
Can models generalize attribute knowledge across semantically and perceptually dissimilar categories? While prior work has addressed attribute prediction within narrow taxonomic or…