25 citations · 135 across the 25 of their papers we have counts for
26 papers
Not Everything That Counts Can Be Counted: A Case for Safe Qualitative AI
Stine Beltoft, Lukas Galke
Artificial intelligence (AI) and large language models (LLM) are reshaping science, with most recent advances culminating in fully-automated scientific discovery pipelines. But qua…
Gumbel-MPNN: Graph Rewiring with Gumbel-Softmax
Marcel Hoffmann, Lukas Galke, Ansgar Scherp
Graph homophily has been considered an essential property for message-passing neural networks (MPNN) in node classification. Recent findings suggest that performance is more closel…
Four Shades of Life Sciences: A Dataset for Disinformation Detection in the Life Sciences
Eva Seidlmayer, Lukas Galke, Konrad U. Förstner
Disseminators of disinformation often seek to attract attention or evoke emotions - typically to gain influence or generate revenue - resulting in distinctive rhetorical patterns t…
A Transformer-based Autoregressive Decoder Architecture for Hierarchical Text Classification
Younes Yousef, Lukas Galke, Ansgar Scherp
Recent approaches in hierarchical text classification (HTC) rely on the capabilities of a pre-trained transformer model and exploit the label semantics and a graph encoder for the…
Efficient Continual Learning for Small Language Models with a Discrete Key-Value Bottleneck
Andor Diera, Lukas Galke, Fabian Karl +1
Continual learning remains a challenge across various natural language processing (NLP) tasks, as models updated with new training data often risk catastrophic forgetting of previo…
Isotropy Matters: Soft-ZCA Whitening of Embeddings for Semantic Code Search
Andor Diera, Lukas Galke, Ansgar Scherp
Low isotropy in an embedding space impairs performance on tasks involving semantic inference. Our study investigates the impact of isotropy on semantic code search performance and…
When are 1.58 bits enough? A Bottom-up Exploration of BitNet Quantization
Jacob Nielsen, Lukas Galke, Peter Schneider-Kamp
Contemporary machine learning models, such as language models, are powerful, but come with immense resource requirements both at training and inference time. It has been shown that…
Your Extreme Multi-label Classifier is Secretly a Hierarchical Text Classifier for Free
Nerijus Bertalis, Paul Granse, Ferhat Gül +6
Assigning a set of labels to a given text is a classification problem with many real-world applications, such as recommender systems. Two separate research streams address this iss…
Tokenization and Morphology in Multilingual Language Models: A Comparative Analysis of mT5 and ByT5
Thao Anh Dang, Limor Raviv, Lukas Galke
Morphology is a crucial factor for multilingual language modeling as it poses direct challenges for tokenization. Here, we seek to understand how tokenization influences the morpho…
POWN: Prototypical Open-World Node Classification
Marcel Hoffmann, Lukas Galke, Ansgar Scherp
We consider the problem of \textit{true} open-world semi-supervised node classification, in which nodes in a graph either belong to known or new classes, with the latter not presen…
Learning and communication pressures in neural networks: Lessons from emergent communication
Lukas Galke, Limor Raviv
Finding and facilitating commonalities between the linguistic behaviors of large language models and humans could lead to major breakthroughs in our understanding of the acquisitio…
GenCodeSearchNet: A Benchmark Test Suite for Evaluating Generalization in Programming Language Understanding
Andor Diera, Abdelhalim Dahou, Lukas Galke +3
Language models can serve as a valuable tool for software developers to increase productivity. Large generative models can be used for code generation and code completion, while sm…
Open-World Lifelong Graph Learning
Marcel Hoffmann, Lukas Galke, Ansgar Scherp
We study the problem of lifelong graph learning in an open-world scenario, where a model needs to deal with new tasks and potentially unknown classes. We utilize Out-of-Distributio…
What makes a language easy to deep-learn? Deep neural networks and humans similarly benefit from compositional structure
Lukas Galke, Yoav Ram, Limor Raviv
Deep neural networks drive the success of natural language processing. A fundamental property of language is its compositional structure, allowing humans to systematically produce…
Emergent Communication for Understanding Human Language Evolution: What's Missing?
Lukas Galke, Yoav Ram, Limor Raviv
Emergent communication protocols among humans and artificial neural network agents do not yet share the same properties and show some critical mismatches in results. We describe th…
Are We Really Making Much Progress in Text Classification? A Comparative Review
Lukas Galke, Ansgar Scherp, Andor Diera +5
We survey the literature on single-label, multi-label, and hierarchical text classification and provide a quantitative comparison of methods categorized into bag-of-words, sequence…
Lifelong Learning on Evolving Graphs Under the Constraints of Imbalanced Classes and New Classes
Lukas Galke, Iacopo Vagliano, Benedikt Franke +3
Lifelong graph learning deals with the problem of continually adapting graph neural network (GNN) models to changes in evolving graphs. We address two critical challenges of lifelo…
General Cross-Architecture Distillation of Pretrained Language Models into Matrix Embeddings
Lukas Galke, Isabelle Cuber, Christoph Meyer +3
Large pretrained language models (PreLMs) are revolutionizing natural language processing across all benchmarks. However, their sheer size is prohibitive for small laboratories or…
Bag-of-Words vs. Graph vs. Sequence in Text Classification: Questioning the Necessity of Text-Graphs and the Surprising Strength of a Wide MLP
Lukas Galke, Ansgar Scherp
Graph neural networks have triggered a resurgence of graph-based text classification methods, defining today's state of the art. We show that a wide multi-layer perceptron (MLP) us…
Recommendations for Item Set Completion: On the Semantics of Item Co-Occurrence With Data Sparsity, Input Size, and Input Modalities
Iacopo Vagliano, Lukas Galke, Ansgar Scherp
We address the problem of recommending relevant items to a user in order to "complete" a partial set of items already known. We consider the two scenarios of citation and subject l…
Lifelong Learning of Graph Neural Networks for Open-World Node Classification
Lukas Galke, Benedikt Franke, Tobias Zielke +1
Graph neural networks (GNNs) have emerged as the standard method for numerous tasks on graph-structured data such as node classification. However, real-world graphs are often evolv…
Multi-Modal Adversarial Autoencoders for Recommendations of Citations and Subject Labels
Lukas Galke, Florian Mai, Iacopo Vagliano +1
We present multi-modal adversarial autoencoders for recommendation and evaluate them on two different tasks: citation recommendation and subject label recommendation. We analyze th…
Can Graph Neural Networks Go "Online"? An Analysis of Pretraining and Inference
Lukas Galke, Iacopo Vagliano, Ansgar Scherp
Large-scale graph data in real-world applications is often not static but dynamic, i. e., new nodes and edges appear over time. Current graph convolution approaches are promising,…
CBOW Is Not All You Need: Combining CBOW with the Compositional Matrix Space Model
Florian Mai, Lukas Galke, Ansgar Scherp
Continuous Bag of Words (CBOW) is a powerful text embedding method. Due to its strong capabilities to encode word content, CBOW embeddings perform well on a wide range of downstrea…