189 citations · 323 across the 46 of their papers we have counts for
18 papers · 1 filter
Memorisation bias in medical AI
Moritz A. Knolle, Martin J. Menten, Laurin Lux +5
Medical AI models hold immense potential to improve patient outcomes, but they are also known to unintentionally memorise individual records from their training datasets. While suc…
Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching
Kristian Schwethelm, Daniel Rueckert, Georgios Kaissis
A main promise of looped language models (LMs) is depth-adaptive inference. By iterating a block of shared layers a variable number of times, the model can use less compute for "ea…
How Much Is One Recurrence Worth? Iso-Depth Scaling Laws for Looped Language Models
Kristian Schwethelm, Daniel Rueckert, Georgios Kaissis
We measure how much one recurrence is worth to a looped (depth-recurrent) transformer, in equivalent unique parameters. From an iso-depth pretraining sweep across recurrence counts…
Unintended Memorization of Sensitive Information in Fine-Tuned Language Models
Marton Szep, Jorge Marin Ruiz, Georgios Kaissis +4
Fine-tuning Large Language Models (LLMs) on sensitive datasets carries a substantial risk of unintended memorization and leakage of Personally Identifiable Information (PII), which…
Sensitivity, Specificity, and Consistency: A Tripartite Evaluation of Privacy Filters for Synthetic Data Generation
Adil Koeken, Alexander Ziller, Moritz Knolle +1
The generation of privacy-preserving synthetic datasets is a promising avenue for overcoming data scarcity in medical AI research. Post-hoc privacy filtering techniques, designed t…
Efficient numeracy in language models through single-token number embeddings
Linus Kreitner, Paul Hager, Jonathan Mengedoht +3
To drive progress in science and engineering, large language models (LLMs) must be able to process large amounts of numerical data and solve long calculations efficiently. This is…