9 citations · 25 across the 13 of their papers we have counts for
14 papers
When Do Biological Reasoning Models Use Their Biological Inputs?
Ada Fang, Nikitha Thoduguli, Lukas Fesser +3
Biological reasoning models use post-training to connect LLMs to biological foundation model representations and biological text. Their benchmark accuracy is taken as evidence that…
How Post-Training Shapes Biological Reasoning Models
Lukas Fesser, Hanlin Zhang, Michelle M. Li +5
Scientific reasoning models for biology combine language models with foundation models trained on multimodal biological data, including DNA, RNA, and proteins. These models are bui…
A Unifying View of Attention Sinks: From Mechanisms to Architectural Interventions
Lukas Fesser, Lukas Fesser*, Mozes Jacobs* +5
When attention concentrates on a single token, a sink, what is the model actually computing? Attention sinks are ubiquitous in softmax transformers, yet this shared visual signatur…
Evaluating Relational Reasoning in LLMs with REL
Lukas Fesser, Yasha Ektefaie, Ada Fang +2
Relational reasoning is the ability to infer relations that jointly bind multiple entities, attributes, or variables. This ability is central to scientific reasoning, but existing…
Performance Heterogeneity in Graph Neural Networks: Lessons for Architecture Design and Preprocessing
Lukas Fesser, Melanie Weber
Graph Neural Networks have emerged as the most popular architecture for graph-level learning, including graph classification and regression tasks, which frequently arise in areas s…
Enhancing the Utility of Higher-Order Information in Relational Learning
Raphael Pellegrin, Lukas Fesser, Melanie Weber
Higher-order information is crucial for relational learning in many domains where relationships extend beyond pairwise interactions. Hypergraphs provide a natural framework for mod…
Multimodal Medical Code Tokenizer
Xiaorui Su, Shvat Messica, Yepeng Huang +5
Foundation models trained on patient electronic health records (EHRs) require tokenizing medical data into sequences of discrete vocabulary items. Existing tokenizers treat medical…
Unitary convolutions for learning on graphs and groups
Bobak T. Kiani, Lukas Fesser, Melanie Weber
Data with geometric structure is ubiquitous in machine learning often arising from fundamental symmetries in a domain, such as permutation-invariance in graphs and translation-inva…
STR-Cert: Robustness Certification for Deep Text Recognition on Deep Learning Pipelines and Vision Transformers
Daqian Shao, Lukas Fesser, Marta Kwiatkowska
Robustness certification, which aims to formally certify the predictions of neural networks against adversarial inputs, has become an integral part of important tool for safety-cri…
Effective Structural Encodings via Local Curvature Profiles
Lukas Fesser, Melanie Weber
Structural and Positional Encodings can significantly improve the performance of Graph Neural Networks in downstream tasks. Recent literature has begun to systematically investigat…
Mitigating Over-Smoothing and Over-Squashing using Augmentations of Forman-Ricci Curvature
Lukas Fesser, Melanie Weber
While Graph Neural Networks (GNNs) have been successfully leveraged for learning on graph-structured data across domains, several potential pitfalls have been described recently. T…
Understanding and Mitigating Extrapolation Failures in Physics-Informed Neural Networks
Lukas Fesser, Luca D'Amico-Wong, Richard Qiu
Physics-informed Neural Networks (PINNs) have recently gained popularity due to their effective approximation of partial differential equations (PDEs) using deep neural networks (D…
Augmentations of Forman's Ricci Curvature and their Applications in Community Detection
Lukas Fesser, Sergio Serrano de Haro Iváñez, Karel Devriendt +2
The notion of curvature on graphs has recently gained traction in the networks community, with the Ollivier-Ricci curvature (ORC) in particular being used for several tasks in netw…
Tightness of Bernoulli Gibbsian line ensembles
Evgeni Dimitrov, Xiang Fang, Lukas Fesser +4
A Bernoulli Gibbsian line ensemble is the law of the trajectories of independent Bernoulli random walkers with possib…