12 papers
Meta-Learning Transformers to Improve In-Context Generalization
Lorenzo Braccaioli, Anna Vettoruzzo, Prabhant Singh +3
In-context learning enables transformer models to generalize to new tasks based solely on input prompts, without any need for weight updates. However, existing training paradigms t…
Croissant Tasks: A Metadata Format for Reproducible Machine Learning Evaluations
Omar Benjelloun, Leonardo Martins Bianco, Isabelle Guyon +8
Reproducibility is fundamental to the scientific method, yet remains a critical challenge in machine learning. Contributing factors include underspecified execution details and bri…
Selective Latent Thinking: Adaptive Compression of LLM Reasoning Chains
Hui Xie, Jie Liu, Ziyue Qiao +1
Explicit chain-of-thought (CoT) reasoning substantially improves the reasoning ability of large language models (LLMs), but incurs high inference cost due to lengthy autoregressive…
Pruned Adaptation Modules: A Simple yet Strong Baseline for Continual Foundation Models
Elif Ceren Gok Yildirim, Murat Onur Yildirim, Joaquin Vanschoren
The continual learning literature has rapidly shifted from traditional class incremental learning (CIL) techniques to foundation model (FM)-based CIL methods without a clear unders…
Unlocking [CLS] Features for Continual Post-Training
Murat Onur Yildirim, Elif Ceren Gok Yildirim, Joaquin Vanschoren
Continual learning requires models to integrate new classes or domains over time while preserving previously acquired knowledge. Within this paradigm, foundation models often achie…
Automated Machine Learning for Unsupervised Tabular Tasks
Prabhant Singh, Pieter Gijsbers, Elif Ceren Gok Yildirim +2
In this work, we present LOTUS (Learning to Learn with Optimal Transport for Unsupervised Scenarios), a simple yet effective method to perform model selection for multiple unsuperv…