9 citations · 25 across the 13 of their papers we have counts for
9 papers · 1 filter
When Do Biological Reasoning Models Use Their Biological Inputs?
Ada Fang, Nikitha Thoduguli, Lukas Fesser +3
Biological reasoning models use post-training to connect LLMs to biological foundation model representations and biological text. Their benchmark accuracy is taken as evidence that…
How Post-Training Shapes Biological Reasoning Models
Lukas Fesser, Hanlin Zhang, Michelle M. Li +5
Scientific reasoning models for biology combine language models with foundation models trained on multimodal biological data, including DNA, RNA, and proteins. These models are bui…
A Unifying View of Attention Sinks: From Mechanisms to Architectural Interventions
Lukas Fesser, Lukas Fesser*, Mozes Jacobs* +5
When attention concentrates on a single token, a sink, what is the model actually computing? Attention sinks are ubiquitous in softmax transformers, yet this shared visual signatur…
Performance Heterogeneity in Graph Neural Networks: Lessons for Architecture Design and Preprocessing
Lukas Fesser, Melanie Weber
Graph Neural Networks have emerged as the most popular architecture for graph-level learning, including graph classification and regression tasks, which frequently arise in areas s…
Enhancing the Utility of Higher-Order Information in Relational Learning
Raphael Pellegrin, Lukas Fesser, Melanie Weber
Higher-order information is crucial for relational learning in many domains where relationships extend beyond pairwise interactions. Hypergraphs provide a natural framework for mod…
Unitary convolutions for learning on graphs and groups
Bobak T. Kiani, Lukas Fesser, Melanie Weber
Data with geometric structure is ubiquitous in machine learning often arising from fundamental symmetries in a domain, such as permutation-invariance in graphs and translation-inva…
Effective Structural Encodings via Local Curvature Profiles
Lukas Fesser, Melanie Weber
Structural and Positional Encodings can significantly improve the performance of Graph Neural Networks in downstream tasks. Recent literature has begun to systematically investigat…
Mitigating Over-Smoothing and Over-Squashing using Augmentations of Forman-Ricci Curvature
Lukas Fesser, Melanie Weber
While Graph Neural Networks (GNNs) have been successfully leveraged for learning on graph-structured data across domains, several potential pitfalls have been described recently. T…
Understanding and Mitigating Extrapolation Failures in Physics-Informed Neural Networks
Lukas Fesser, Luca D'Amico-Wong, Richard Qiu
Physics-informed Neural Networks (PINNs) have recently gained popularity due to their effective approximation of partial differential equations (PDEs) using deep neural networks (D…