15 papers
Flash PD-SSM: Memory-Optimized Structured Sparse State-Space Models
Aleksandar TerziÄ, Francesco Carzaniga, Nicolas Menet +4
State-space models (SSMs) face a fundamental trade-off between efficiency and expressivity that is mainly dictated by the structure of the model's transition matrix. Unstructured t…
Locally Coherent Parallel Decoding in Diffusion Language Models
Michael Hersche, Nicolas Menet, Ronan Tanios +1
Diffusion language models (DLMs) have emerged as a promising alternative to autoregressive (AR) models, offering sub-linear generation latency and bidirectional capabilities that a…
A Theoretical Analysis of Test-Driven Code Generation
Nicolas Menet, Michael Hersche, Andreas Krause +1
Code assistants are increasingly utilized in test-driven software development, yet the theoretical mechanisms behind their environment-interaction strategies remain underexplored.…
Soft-Masked Diffusion Language Models
Michael Hersche, Samuel Moor-Smith, Thomas Hofmann +1
Diffusion models have demonstrated strong potential in language modeling, offering various advantages over traditional autoregressive approaches. Their ability to generate and revi…
Thompson Sampling via Fine-Tuning of LLMs
Nicolas Menet, Aleksandar TerziÄ, Michael Hersche +2
Bayesian optimization in large unstructured discrete spaces is often hindered by the computational cost of maximizing acquisition functions due to the absence of gradients. We prop…
A Composable Channel-Adaptive Architecture for Seizure Classification
Francesco Carzaniga, Michael Hersche, Kaspar Schindler +1
Objective: We develop a channel-adaptive (CA) architecture that seamlessly processes multi-variate time-series with an arbitrary number of channels, and in particular intracranial…