6 papers
Quantum Doubly Stochastic Transformers
Jannis Born, Filip Skogh, Kahn Rhrissorrakrai +3
At the core of the Transformer, the softmax normalizes the attention matrix to be right stochastic. Previous research has shown that this often de-stabilizes training and that enfo…
COVID-BLUeS -- A Prospective Study on the Value of AI in Lung Ultrasound Analysis
Nina Wiedemann, Dianne de Korte-de Boer, Matthias Richter +10
As a lightweight and non-invasive imaging technique, lung ultrasound (LUS) has gained importance for assessing lung pathologies. The use of Artificial intelligence (AI) in medical…
Regress, Don't Guess -- A Regression-like Loss on Number Tokens for Language Models
Jonas Zausinger, Lars Pennig, Anamarija Kozina +13
While language models have exceptional capabilities at text generation, they lack a natural inductive bias for emitting numbers and thus struggle in tasks involving quantitative re…
Towards generalizable single-cell perturbation modeling via the Conditional Monge Gap
Alice Driessen, Benedek Harsanyi, Marianna Rapsomaniki +1
Learning the response of single-cells to various treatments offers great potential to enable targeted therapies. In this context, neural optimal transport (OT) has emerged as a pri…
We Need Improved Data Curation and Attribution in AI for Scientific Discovery
Mara Graziani, Antonio Foncubierta, Dimitrios Christofidellis +5
As the interplay between human-generated and synthetic data evolves, new challenges arise in scientific discovery concerning the integrity of the data and the stability of the mode…
Multiscale Byte Language Models -- A Hierarchical Architecture for Causal Million-Length Sequence Modeling
Eric Egli, Matteo Manica, Jannis Born
Bytes form the basis of the digital world and thus are a promising building block for multimodal foundation models. Recently, Byte Language Models (BLMs) have emerged to overcome t…