6 papers
Do Language Models Encode Semantic Relations? Probing and Sparse Feature Analysis
Andor Diera, Ansgar Scherp
Understanding whether large language models (LLMs) capture structured meaning requires examining how they represent concept relationships. In this work, we study three models of in…
Efficient Continual Learning for Small Language Models with a Discrete Key-Value Bottleneck
Andor Diera, Lukas Galke, Fabian Karl +1
Continual learning remains a challenge across various natural language processing (NLP) tasks, as models updated with new training data often risk catastrophic forgetting of previo…
Are We Really Making Much Progress in Text Classification? A Comparative Review
Lukas Galke, Ansgar Scherp, Andor Diera +5
We analyze various methods for single-label and multi-label text classification across well-known datasets, categorizing them into bag-of-words, sequence-based, graph-based, and hi…
Multi-View Structural Graph Summaries
Jonatan Frank, Andor Diera, David Richerby +1
A structural graph summary is a small graph representation that preserves structural information necessary for a given task. The summary is used instead of the original graph to co…
Memorization of Named Entities in Fine-tuned BERT Models
Andor Diera, Nicolas Lell, Aygul Garifullina +1
Privacy preserving deep learning is an emerging field in machine learning that aims to mitigate the privacy risks in the use of deep neural networks. One such risk is training data…
Isotropy Matters: Soft-ZCA Whitening of Embeddings for Semantic Code Search
Andor Diera, Lukas Galke, Ansgar Scherp
Low isotropy in an embedding space impairs performance on tasks involving semantic inference. Our study investigates the impact of isotropy on semantic code search performance and…