15 papers
Post-Training Speech Enhancement Language Models with Perceptual Rewards
Frédéric Berdoz, Luca A. Lanzendörfer, Antonis Asonitis +1
Speech enhancement language models achieve strong results when trained on discrete audio tokens, but their optimization relies on token-level cross-entropy rather than the perceptu…
Data Attribution in Large Language Models via Bidirectional Gradient Optimization
Frédéric Berdoz, Luca A. Lanzendörfer, Kaan Bayraktar +1
Large Language Models (LLMs) are increasingly deployed across diverse applications, raising critical questions for governance, accountability, and data provenance. Understanding wh…
Reasoning Structure of Large Language Models
Frédéric Berdoz, Luca A. Lanzendörfer, Fabian Farestam +1
Large reasoning models (LRMs) are often evaluated using metrics such as final-answer accuracy or token count. However, identical scores on these metrics can hide fundamentally diff…
TreeFlash: Parallel AR-Approximation for Faster Speculative Decoding
Peer Rheinboldt, Frédéric Berdoz, Roger Wattenhofer
One-shot block drafters for speculative decoding generate the full draft in a single forward pass, achieving strong throughput by eliminating sequential token generation. However,…
Alignment-Aware Decoding
Frédéric Berdoz, Luca A. Lanzendörfer, René Caky +1
Alignment of large language models remains a central challenge in natural language processing. Preference optimization has emerged as a popular and effective method for improving a…
N-vium: Mixture-of-Exits Transformer for Accelerated Exact Generation
Aleksander Lorenc, Frédéric Berdoz, Joël Mathys +1
Improving the inference efficiency of autoregressive transformers typically means reducing FLOPs per token, usually through approximations that degrade model quality. We introduce…