8 papers · 1 filter
Efficient Continual Learning for Small Language Models with a Discrete Key-Value Bottleneck
Andor Diera, Lukas Galke, Fabian Karl +1
Continual learning remains a challenge across various natural language processing (NLP) tasks, as models updated with new training data often risk catastrophic forgetting of previo…
Your Extreme Multi-label Classifier is Secretly a Hierarchical Text Classifier for Free
Nerijus Bertalis, Paul Granse, Ferhat Gül +6
Assigning a set of labels to a given text is a classification problem with many real-world applications, such as recommender systems. Two separate research streams address this iss…
Four Shades of Life Sciences: A Dataset for Disinformation Detection in the Life Sciences
Eva Seidlmayer, Lukas Galke, Konrad U. Förstner
Disseminators of disinformation often seek to attract attention or evoke emotions - typically to gain influence or generate revenue - resulting in distinctive rhetorical patterns t…
Are We Really Making Much Progress in Text Classification? A Comparative Review
Lukas Galke, Ansgar Scherp, Andor Diera +5
We analyze various methods for single-label and multi-label text classification across well-known datasets, categorizing them into bag-of-words, sequence-based, graph-based, and hi…
Isotropy Matters: Soft-ZCA Whitening of Embeddings for Semantic Code Search
Andor Diera, Lukas Galke, Ansgar Scherp
Low isotropy in an embedding space impairs performance on tasks involving semantic inference. Our study investigates the impact of isotropy on semantic code search performance and…
Learning and communication pressures in neural networks: Lessons from emergent communication
Lukas Galke, Limor Raviv
Finding and facilitating commonalities between the linguistic behaviors of large language models and humans could lead to major breakthroughs in our understanding of the acquisitio…