7 papers · 1 filter
Post-Training Speech Enhancement Language Models with Perceptual Rewards
Frédéric Berdoz, Luca A. Lanzendörfer, Antonis Asonitis +1
Speech enhancement language models achieve strong results when trained on discrete audio tokens, but their optimization relies on token-level cross-entropy rather than the perceptu…
Data Attribution in Large Language Models via Bidirectional Gradient Optimization
Frédéric Berdoz, Luca A. Lanzendörfer, Kaan Bayraktar +1
Large Language Models (LLMs) are increasingly deployed across diverse applications, raising critical questions for governance, accountability, and data provenance. Understanding wh…
TreeFlash: Parallel AR-Approximation for Faster Speculative Decoding
Peer Rheinboldt, Frédéric Berdoz, Roger Wattenhofer
One-shot block drafters for speculative decoding generate the full draft in a single forward pass, achieving strong throughput by eliminating sequential token generation. However,…
Alignment-Aware Decoding
Frédéric Berdoz, Luca A. Lanzendörfer, René Caky +1
Alignment of large language models remains a central challenge in natural language processing. Preference optimization has emerged as a popular and effective method for improving a…
N-vium: Mixture-of-Exits Transformer for Accelerated Exact Generation
Aleksander Lorenc, Frédéric Berdoz, Joël Mathys +1
Improving the inference efficiency of autoregressive transformers typically means reducing FLOPs per token, usually through approximations that degrade model quality. We introduce…
Subliminal Signals in Preference Labels
Isotta Magistrali, Frédéric Berdoz, Sam Dauncey +1
As AI systems approach superhuman capabilities, scalable oversight increasingly relies on LLM-as-a-judge frameworks where models evaluate and guide each other's training. A core as…