13 papers
Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching
Kristian Schwethelm, Daniel Rueckert, Georgios Kaissis
A main promise of looped language models (LMs) is depth-adaptive inference. By iterating a block of shared layers a variable number of times, the model can use less compute for "ea…
Addressing Benchmarking Gaps in Large Language Models for Health and Medicine with Dynamic Red-Teaming
Jiazhen Pan, Bailiang Jian, Paul Hager +19
The paper presents a dynamic red‑teaming framework (DAS) that continuously stress‑tests large language models on health tasks for robustness, privacy, bias, and hallucination, reve…
Efficient numeracy in language models through single-token number embeddings
Linus Kreitner, Paul Hager, Jonathan Mengedoht +3
To drive progress in science and engineering, large language models (LLMs) must be able to process large amounts of numerical data and solve long calculations efficiently. This is…
Survival In-Context: Amortized Bayesian Survival Analysis via Prior-Fitted Networks
Dmitrii Seletkov, Paul Hager, Georgios Kaissis +3
Survival analysis is crucial for many medical applications, but remains challenging for modern machine learning due to limited data, censoring, and the heterogeneity of tabular cov…
How Much Is One Recurrence Worth? Iso-Depth Scaling Laws for Looped Language Models
Kristian Schwethelm, Daniel Rueckert, Georgios Kaissis
We measure how much one recurrence is worth to a looped (depth-recurrent) transformer, in equivalent unique parameters. From an iso-depth pretraining sweep across recurrence counts…
On Arbitrary Predictions from Equally Valid Models
Sarah Lockfisch, Kristian Schwethelm, Martin Menten +4
Model multiplicity refers to the existence of multiple machine learning models that describe the data equally well but may produce different predictions on individual samples. In m…