23 papers · 1 filter
Persistent Sparse Autoencoders: Learning Feature Timescales in Language Models
Haoyan Luo, Mateo Espinosa Zarlenga, Mateja Jamnik
Sparse autoencoders (SAEs) decompose language model activations into sparse features, but standard SAEs encode each token independently and do not expose information that persists…
KnowsTFM: Knowledge-Informed Fine-Tuning of Small Tabular Foundation Models
Boshko Koloski, Xiangjian Jiang, Senja Pollak +3
Tabular foundation models have advanced deep learning for tabular data by delivering strong default performance across many small and medium tasks. Yet in niche domains, where data…
The Standard Interpretable Model: A general theory of interpretable machine learning to deductively design interpretable methods using Lagrangian mechanics
Pietro Barbiero, Giovanni De Felice, Mateo Espinosa Zarlenga +5
As Artificial Intelligence models grow in complexity, interpretability has become an indispensable tool for understanding, debugging, and controlling their computations. However, i…
CB-SLICE: Concept-Based Interpretable Error Slice Discovery
Yael Konforti, Mateo Espinosa Zarlenga, Elaf Almahmoud +1
Despite strong average-case performance, deep learning models often exhibit systematic errors on specific population groups, known as error slices. Identifying these groups and the…
Training Language Agents to Learn from Experience
Yuval Shalev, Zifeng Ding, Mateja Jamnik
Language agents can adapt from experience in interactive environments, but current reflection-based methods can only self-correct within a single task instance. Whether such experi…
Tabular Foundation Model for Generative Modelling
Xiangjian Jiang, Mingxuan Liu, Nikola Simidjievski +2
Generative modelling is a demanding test of foundation models, because it requires robust, holistic representation learning for a given data modality, rather than optimisation for…