7 papers
Embedding World Knowledge into Tabular Models: Towards Best Practices for Embedding Pipeline Design
Oksana Kolomenko, Ricardo Knauer, Erik Rodner
Embeddings are a powerful way to enrich data-driven machine learning models with the world knowledge of large language models (LLMs). Yet, there is limited evidence on how to desig…
Robust Weight Imprinting: Insights from Neural Collapse and Proxy-Based Aggregation
Justus Westerhoff, Golzar Atefi, Mario Koddenbrock +4
The capacity of foundation models allows for their application to new, unseen tasks. The adaptation to such tasks is called transfer learning. An efficient transfer learning method…
In Search of Grandmother Cells: Tracing Interpretable Neurons in Tabular Representations
Ricardo Knauer, Erik Rodner
Foundation models are powerful yet often opaque in their decision-making. A topic of continued interest in both neuroscience and artificial intelligence is whether some neurons beh…
Is Visual in-Context Learning for Compositional Medical Tasks within Reach?
Simon ReiÃ, Zdravko Marinov, Alexander Jaus +4
In this paper, we explore the potential of visual in-context learning to enable a single model to handle multiple tasks and adapt to new tasks during test time without re-training.…
On the Domain Robustness of Contrastive Vision-Language Models
Mario Koddenbrock, Rudolf Hoffmann, David Brodmann +1
In real-world vision-language applications, practitioners increasingly rely on large, pretrained foundation models rather than custom-built solutions, despite limited transparency…
"Oh LLM, I'm Asking Thee, Please Give Me a Decision Tree": Zero-Shot Decision Tree Induction and Embedding with Large Language Models
Ricardo Knauer, Mario Koddenbrock, Raphael Wallsberger +5
Large language models (LLMs) provide powerful means to leverage prior knowledge for predictive modeling when data is limited. In this work, we demonstrate how LLMs can use their co…