7 papers
Fine-Tuning Regimes Define Distinct Continual Learning Problems
Paul-Tiberiu Iordache, Elena Burceanu
Continual learning (CL) studies how models acquire tasks sequentially while retaining previously learned knowledge. Despite substantial progress in benchmarking CL methods, compara…
idSCD: Identifying Training Datasets through Semantic Correlation Descriptors
Andrada Gobeaja, Ionut Hodoroaga, Elena Burceanu +1
Can a dataset be recognized from the spurious correlations it induces during training? We argue that datasets leave dataset-specific traces in a model's learned semantic correlatio…
JumpLoRA: Sparse Adapters for Continual Learning in Large Language Models
Alexandra Dragomir, Ioana Pintilie, Antonio Barbalau +6
Adapter-based methods have become a cost-effective approach to continual learning (CL) for Large Language Models (LLMs), by sequentially learning a low-rank update matrix for each…
Temporal Taskification in Streaming Continual Learning: A Source of Evaluation Instability
Nicolae Filat, Ahmed Hussain, Konstantinos Kalogiannis +1
Streaming Continual Learning (CL) typically converts a continuous stream into a sequence of discrete tasks through temporal partitioning. We argue that this temporal taskification…
Closing the gap on tabular data with Fourier and Implicit Categorical Features
Marius Dragoi, Florin Gogianu, Elena Burceanu
While Deep Learning has demonstrated impressive results in applications on various data types, it continues to lag behind tree-based methods when applied to tabular data, often ref…
Rethinking Sparse Autoencoders: Select-and-Project for Fairness and Control from Encoder Features Alone
Antonio BÄrbÄlau, Cristian Daniel PÄduraru, Teodor Poncu +2
Sparse Autoencoders (SAEs) are widely employed for mechanistic interpretability and model steering. Within this context, steering is by design performed by means of decoding altere…