25 citations · 135 across the 25 of their papers we have counts for
13 papers · 1 filter
Four Shades of Life Sciences: A Dataset for Disinformation Detection in the Life Sciences
Eva Seidlmayer, Lukas Galke, Konrad U. Förstner
Disseminators of disinformation often seek to attract attention or evoke emotions - typically to gain influence or generate revenue - resulting in distinctive rhetorical patterns t…
Efficient Continual Learning for Small Language Models with a Discrete Key-Value Bottleneck
Andor Diera, Lukas Galke, Fabian Karl +1
Continual learning remains a challenge across various natural language processing (NLP) tasks, as models updated with new training data often risk catastrophic forgetting of previo…
Isotropy Matters: Soft-ZCA Whitening of Embeddings for Semantic Code Search
Andor Diera, Lukas Galke, Ansgar Scherp
Low isotropy in an embedding space impairs performance on tasks involving semantic inference. Our study investigates the impact of isotropy on semantic code search performance and…
Your Extreme Multi-label Classifier is Secretly a Hierarchical Text Classifier for Free
Nerijus Bertalis, Paul Granse, Ferhat Gül +6
Assigning a set of labels to a given text is a classification problem with many real-world applications, such as recommender systems. Two separate research streams address this iss…
Tokenization and Morphology in Multilingual Language Models: A Comparative Analysis of mT5 and ByT5
Thao Anh Dang, Limor Raviv, Lukas Galke
Morphology is a crucial factor for multilingual language modeling as it poses direct challenges for tokenization. Here, we seek to understand how tokenization influences the morpho…
Learning and communication pressures in neural networks: Lessons from emergent communication
Lukas Galke, Limor Raviv
Finding and facilitating commonalities between the linguistic behaviors of large language models and humans could lead to major breakthroughs in our understanding of the acquisitio…
GenCodeSearchNet: A Benchmark Test Suite for Evaluating Generalization in Programming Language Understanding
Andor Diera, Abdelhalim Dahou, Lukas Galke +3
Language models can serve as a valuable tool for software developers to increase productivity. Large generative models can be used for code generation and code completion, while sm…
What makes a language easy to deep-learn? Deep neural networks and humans similarly benefit from compositional structure
Lukas Galke, Yoav Ram, Limor Raviv
Deep neural networks drive the success of natural language processing. A fundamental property of language is its compositional structure, allowing humans to systematically produce…
Emergent Communication for Understanding Human Language Evolution: What's Missing?
Lukas Galke, Yoav Ram, Limor Raviv
Emergent communication protocols among humans and artificial neural network agents do not yet share the same properties and show some critical mismatches in results. We describe th…
Are We Really Making Much Progress in Text Classification? A Comparative Review
Lukas Galke, Ansgar Scherp, Andor Diera +5
We survey the literature on single-label, multi-label, and hierarchical text classification and provide a quantitative comparison of methods categorized into bag-of-words, sequence…
General Cross-Architecture Distillation of Pretrained Language Models into Matrix Embeddings
Lukas Galke, Isabelle Cuber, Christoph Meyer +3
Large pretrained language models (PreLMs) are revolutionizing natural language processing across all benchmarks. However, their sheer size is prohibitive for small laboratories or…
Bag-of-Words vs. Graph vs. Sequence in Text Classification: Questioning the Necessity of Text-Graphs and the Surprising Strength of a Wide MLP
Lukas Galke, Ansgar Scherp
Graph neural networks have triggered a resurgence of graph-based text classification methods, defining today's state of the art. We show that a wide multi-layer perceptron (MLP) us…
CBOW Is Not All You Need: Combining CBOW with the Compositional Matrix Space Model
Florian Mai, Lukas Galke, Ansgar Scherp
Continuous Bag of Words (CBOW) is a powerful text embedding method. Due to its strong capabilities to encode word content, CBOW embeddings perform well on a wide range of downstrea…