75 citations · 394 across the 149 of their papers we have counts for
54 papers · 1 filter
Post-Training Speech Enhancement Language Models with Perceptual Rewards
Frédéric Berdoz, Luca A. Lanzendörfer, Antonis Asonitis +1
Speech enhancement language models achieve strong results when trained on discrete audio tokens, but their optimization relies on token-level cross-entropy rather than the perceptu…
Data Attribution in Large Language Models via Bidirectional Gradient Optimization
Frédéric Berdoz, Luca A. Lanzendörfer, Kaan Bayraktar +1
Large Language Models (LLMs) are increasingly deployed across diverse applications, raising critical questions for governance, accountability, and data provenance. Understanding wh…
TreeFlash: Parallel AR-Approximation for Faster Speculative Decoding
Peer Rheinboldt, Frédéric Berdoz, Roger Wattenhofer
One-shot block drafters for speculative decoding generate the full draft in a single forward pass, achieving strong throughput by eliminating sequential token generation. However,…
N-vium: Mixture-of-Exits Transformer for Accelerated Exact Generation
Aleksander Lorenc, Frédéric Berdoz, Joël Mathys +1
Improving the inference efficiency of autoregressive transformers typically means reducing FLOPs per token, usually through approximations that degrade model quality. We introduce…
From Message-Passing to Linearized Graph Sequence Models
Joël Mathys, Basil Rohner, Saku Peltonen +1
Message-passing based approaches form the default backbone of most learning architectures on graph-structured data. However, the rapid progress of modern deep learning architecture…
Subliminal Signals in Preference Labels
Isotta Magistrali, Frédéric Berdoz, Sam Dauncey +1
As AI systems approach superhuman capabilities, scalable oversight increasingly relies on LLM-as-a-judge frameworks where models evaluate and guide each other's training. A core as…
On the Expressive Power of GNNs for Boolean Satisfiability
Saku Peltonen, Roger Wattenhofer
Machine learning approaches to solving Boolean Satisfiability (SAT) aim to replace handcrafted heuristics with learning-based models. Graph Neural Networks have emerged as the main…
You Can Learn Tokenization End-to-End with Reinforcement Learning
Sam Dauncey, Roger Wattenhofer
Tokenization is a hardcoded compression step which remains in the training pipeline of Large Language Models (LLMs), despite a general trend towards architectures becoming increasi…
BitLogic: Training Framework for Gradient-Based FPGA-Native Neural Networks
Simon Bührer, Andreas Plesner, Aczel Till +1
Gradient-based LUT- and logic-gate-based neural networks (LUTNet, LogicNets, DiffLogic, PolyLUT, NeuraLUT, WARP-LUT, DWN, LILogicNet, LightLUT) replace multiply-accumulate arithmet…
EEG-Bench: A Benchmark for EEG Foundation Models in Clinical Applications
Ard Kastrati, Josua Bürki, Jonas Lauer +3
We introduce a unified benchmarking framework focused on evaluating EEG-based foundation models in clinical applications. The benchmark spans 11 well-defined diagnostic tasks acros…
Adapting Neural Audio Codecs to EEG
Ard Kastrati, Luca Lanzendörfer, Riccardo Rigoni +2
EEG and audio are inherently distinct modalities, differing in sampling rate, channel structure, and scale. Yet, we show that pretrained neural audio codecs can serve as effective…
Steering Pretrained Drafters during Speculative Decoding
Frédéric Berdoz, Peer Rheinboldt, Roger Wattenhofer
Speculative decoding accelerates language model inference by separating generation into fast drafting and parallel verification. Its main limitation is drafter-verifier misalignmen…
Scalable Evaluation and Neural Models for Compositional Generalization
Giacomo Camposampiero, Pietro Barbiero, Michael Hersche +2
Compositional generalization-a key open challenge in modern machine learning-requires models to predict unknown combinations of known concepts. However, assessing compositional gen…
I-RAVEN-X: Benchmarking Generalization and Robustness of Analogical and Mathematical Reasoning in Large Language and Reasoning Models
Giacomo Camposampiero, Michael Hersche, Roger Wattenhofer +2
We introduce I-RAVEN-X, a symbolic benchmark designed to evaluate generalization and robustness in analogical and mathematical reasoning for Large Language Models (LLMs) and Large…
Inductive Transfer Learning for Graph-Based Recommenders
Florian Grötschla, Elia Trachsel, Luca A. Lanzendörfer +1
Graph-based recommender systems are commonly trained in transductive settings, which limits their applicability to new users, items, or datasets. We propose NBF-Rec, a graph-based…
Efficient Bayesian Inference from Noisy Pairwise Comparisons
Till Aczel, Lucas Theis, Roger Wattenhofer
Evaluating generative models is challenging because standard metrics often fail to reflect human preferences. Human evaluations are more reliable but costly and noisy, as participa…
Parametric Neural Amp Modeling with Active Learning
Florian Grötschla, Longxiang Jiao, Luca A. Lanzendörfer +1
We introduce Panama, an active learning framework to train parametric guitar amp models end-to-end using a combination of an LSTM model and a WaveNet-like architecture. With \model…
From MNIST to ImageNet: Understanding the Scalability Boundaries of Differentiable Logic Gate Networks
Sven Brändle, Till Aczel, Andreas Plesner +1
Differentiable Logic Gate Networks (DLGNs) are a very fast and energy-efficient alternative to conventional feed-forward networks. With learnable combinations of logical gates, DLG…
Light Differentiable Logic Gate Networks
Lukas Rüttgers, Till Aczel, Andreas Plesner +1
Differentiable logic gate networks (DLGNs) exhibit extraordinary efficiency at inference while sustaining competitive accuracy. But vanishing gradients, discretization errors, and…
Alignment-Aware Decoding
Frédéric Berdoz, Luca A. Lanzendörfer, René Caky +1
Alignment of large language models remains a central challenge in natural language processing. Preference optimization has emerged as a popular and effective method for improving a…
Recurrent Deep Differentiable Logic Gate Networks
Simon Bührer, Andreas Plesner, Till Aczel +1
While differentiable logic gates have shown promise in feedforward networks, their application to sequential modeling remains unexplored. This paper presents the first implementati…
Parametric Neural Amp Modeling with Active Learning
Florian Grötschla, Luca A. Lanzendörfer, Longxiang Jiao +1
We introduce PANAMA, an active learning framework for the training of end-to-end parametric guitar amp models using a WaveNet-like architecture. With \model, one can create a virtu…
Benchmarking Music Generation Models and Metrics via Human Preference Studies
Florian Grötschla, Ahmet Solak, Luca A. Lanzendörfer +1
Recent advancements have brought generated music closer to human-created compositions, yet evaluating these models remains challenging. While human preference is the gold standard…
Beyond Interpolation: Extrapolative Reasoning with Reinforcement Learning and Graph Neural Networks
Niccolò Grillo, Andrea Toccaceli, Joël Mathys +3
Despite incredible progress, many neural architectures fail to properly generalize beyond their training distribution. As such, learning to reason in a correct and generalizable wa…