6 papers
JumpLoRA: Sparse Adapters for Continual Learning in Large Language Models
Alexandra Dragomir, Ioana Pintilie, Antonio Barbalau +6
Adapter-based methods have become a cost-effective approach to continual learning (CL) for Large Language Models (LLMs), by sequentially learning a low-rank update matrix for each…
Rethinking Sparse Autoencoders: Select-and-Project for Fairness and Control from Encoder Features Alone
Antonio Bărbălau, Cristian Daniel Păduraru, Teodor Poncu +2
Sparse Autoencoders (SAEs) are widely employed for mechanistic interpretability and model steering. Within this context, steering is by design performed by means of decoding altere…
Fairness for the People, by the People: Minority Collective Action
Omri Ben-Dov, Samira Samadi, Amartya Sanyal +1
Machine learning models often preserve biases present in training data, leading to unfair treatment of certain minority groups. Despite an array of existing firm-side bias mitigati…
Learning Pareto manifolds in high dimensions: How can regularization help?
Tobias Wegel, Filip Kovačević, Alexandru Ţifrea +1
Simultaneously addressing multiple objectives is becoming increasingly important in modern machine learning. At the same time, data is often high-dimensional and costly to label. F…
FRAPPE: A Group Fairness Framework for Post-Processing Everything
Alexandru Tifrea, Preethi Lahoti, Ben Packer +3
Despite achieving promising fairness-error trade-offs, in-processing mitigation techniques for group fairness cannot be employed in numerous practical applications with limited com…
Poincaré GloVe: Hyperbolic Word Embeddings
Alexandru Tifrea, Gary Bécigneul, Octavian-Eugen Ganea
Words are not created equal. In fact, they form an aristocratic graph with a latent hierarchical structure that the next generation of unsupervised learned word embeddings should r…