3 papers
cs.LG2026
TACTICL: Task-Aware Compression of Tabular ICL Models
Mykhailo Koshil, Matthias Feurer, Katharina Eggensperger
The strong performance of foundation models for tabular tasks comes at substantial inference costs. Distilling models into task-specific architectures reduces model size and comput…
cs.LG2026
Is One Layer Enough? Understanding Inference Dynamics in Tabular Foundation Models
Amir Rezaei Balef, Mykhailo Koshil, Katharina Eggensperger
Transformer-based tabular foundation models (TFMs) dominate small to medium tabular predictive benchmark tasks, yet their inference mechanisms remain largely unexplored. We present…
cs.LG2025
Towards Understanding Layer Contributions in Tabular In-Context Learning Models
Amir Rezaei Balef, Mykhailo Koshil, Katharina Eggensperger
Despite the architectural similarities between tabular in-context learning (ICL) models and large language models (LLMs), little is known about how individual layers contribute to…