8 citations · 10 across the 25 of their papers we have counts for
27 papers
Flash PD-SSM: Memory-Optimized Structured Sparse State-Space Models
Aleksandar Terzić, Francesco Carzaniga, Nicolas Menet +4
State-space models (SSMs) face a fundamental trade-off between efficiency and expressivity that is mainly dictated by the structure of the model's transition matrix. Unstructured t…
POETS: Uncertainty-Aware LLM Optimization via Compute-Efficient Policy Ensembles
Nicolas Menet, Andreas Krause, Abbas Rahimi
Balancing exploration with exploitation is a core challenge in black-box optimization and reinforcement learning. We introduce POETS (licy nsembles for $\t…
Locally Coherent Parallel Decoding in Diffusion Language Models
Michael Hersche, Nicolas Menet, Ronan Tanios +1
Diffusion language models (DLMs) have emerged as a promising alternative to autoregressive (AR) models, offering sub-linear generation latency and bidirectional capabilities that a…
A Theoretical Analysis of Test-Driven Code Generation
Nicolas Menet, Michael Hersche, Andreas Krause +1
Code assistants are increasingly utilized in test-driven software development, yet the theoretical mechanisms behind their environment-interaction strategies remain underexplored.…
A Composable Channel-Adaptive Architecture for Seizure Classification
Francesco Carzaniga, Michael Hersche, Kaspar Schindler +1
Objective: We develop a channel-adaptive (CA) architecture that seamlessly processes multi-variate time-series with an arbitrary number of channels, and in particular intracranial…
Scalable Evaluation and Neural Models for Compositional Generalization
Giacomo Camposampiero, Pietro Barbiero, Michael Hersche +2
Compositional generalization-a key open challenge in modern machine learning-requires models to predict unknown combinations of known concepts. However, assessing compositional gen…
I-RAVEN-X: Benchmarking Generalization and Robustness of Analogical and Mathematical Reasoning in Large Language and Reasoning Models
Giacomo Camposampiero, Michael Hersche, Roger Wattenhofer +2
We introduce I-RAVEN-X, a symbolic benchmark designed to evaluate generalization and robustness in analogical and mathematical reasoning for Large Language Models (LLMs) and Large…
Soft-Masked Diffusion Language Models
Michael Hersche, Samuel Moor-Smith, Thomas Hofmann +1
Diffusion models have demonstrated strong potential in language modeling, offering various advantages over traditional autoregressive approaches. Their ability to generate and revi…
Thompson Sampling via Fine-Tuning of LLMs
Nicolas Menet, Aleksandar Terzić, Michael Hersche +2
Bayesian optimization in large unstructured discrete spaces is often hindered by the computational cost of maximizing acquisition functions due to the absence of gradients. We prop…
Structured Sparse Transition Matrices to Enable State Tracking in State-Space Models
Aleksandar Terzić, Nicolas Menet, Michael Hersche +2
Modern state-space models (SSMs) often utilize transition matrices which enable efficient computation but pose restrictions on the model's expressivity, as measured in terms of the…
A foundation model with multi-variate parallel attention to generate neuronal activity
Francesco Carzaniga, Michael Hersche, Abu Sebastian +2
Learning from multi-variate time-series with heterogeneous channel configurations remains a fundamental challenge for deep neural networks, particularly in clinical domains such as…
Analog Foundation Models
Julian Büchel, Iason Chalas, Giovanni Acampa +7
Analog in-memory computing (AIMC) is a promising compute paradigm to improve speed and power efficiency of neural network inference beyond the limits of conventional von Neumann-ba…
Can Large Reasoning Models do Analogical Reasoning under Perceptual Uncertainty?
Giacomo Camposampiero, Michael Hersche, Roger Wattenhofer +2
This work presents a first evaluation of two state-of-the-art Large Reasoning Models (LRMs), OpenAI's o3-mini and DeepSeek R1, on analogical reasoning, focusing on well-established…
The Case for Cleaner Biosignals: High-fidelity Neural Compressor Enables Transfer from Cleaner iEEG to Noisier EEG
Francesco Stefano Carzaniga, Gary Tom Hoppeler, Michael Hersche +2
All data modalities are not created equal, even when the signal they measure comes from the same source. In the case of the brain, two of the most important data modalities are the…
Towards Learning to Reason: Comparing LLMs with Neuro-Symbolic on Arithmetic Relations in Abstract Reasoning
Michael Hersche, Giacomo Camposampiero, Roger Wattenhofer +2
This work compares large language models (LLMs) and neuro-symbolic approaches in solving Raven's progressive matrices (RPM), a visual abstract reasoning test that involves the unde…
On the Expressiveness and Length Generalization of Selective State-Space Models on Regular Languages
Aleksandar Terzić, Michael Hersche, Giacomo Camposampiero +3
Selective state-space models (SSMs) are an emerging alternative to the Transformer, offering the unique advantage of parallel training and sequential inference. Although these mode…
On the Role of Noise in Factorizers for Disentangling Distributed Representations
Geethan Karunaratne, Michael Hersche, Abu Sebastian +1
To efficiently factorize high-dimensional distributed representations to the constituent atomic vectors, one can exploit the compute-in-superposition capabilities of vector-symboli…
Kernel Approximation using Analog In-Memory Computing
Julian Büchel, Giacomo Camposampiero, Athanasios Vasilopoulos +4
Kernel functions are vital ingredients of several machine learning algorithms, but often incur significant memory and computational costs. We introduce an approach to kernel approx…
Retro-li: Small-Scale Retrieval Augmented Generation Supporting Noisy Similarity Searches and Domain Shift Generalization
Gentiana Rashiti, Geethan Karunaratne, Mrinmaya Sachan +2
The retrieval augmented generation (RAG) system such as Retro has been shown to improve language modeling capabilities and reduce toxicity and hallucinations by retrieving from a d…
Terminating Differentiable Tree Experts
Jonathan Thomm, Michael Hersche, Giacomo Camposampiero +3
We advance the recently proposed neuro-symbolic Differentiable Tree Machine, which learns tree operations using a combination of transformers and Tensor Product Representations. We…
Towards Learning Abductive Reasoning using VSA Distributed Representations
Giacomo Camposampiero, Michael Hersche, Aleksandar Terzić +3
We introduce the Abductive Rule Learner with Context-awareness (ARLC), a model that solves abstract reasoning tasks based on Learn-VRF. ARLC features a novel and more broadly appli…
12 mJ per Class On-Device Online Few-Shot Class-Incremental Learning
Yoga Esa Wibowo, Cristian Cioflan, Thorir Mar Ingolfsson +4
Few-Shot Class-Incremental Learning (FSCIL) enables machine learning systems to expand their inference capabilities to new classes using only a few labeled examples, without forget…
Limits of Transformer Language Models on Learning to Compose Algorithms
Jonathan Thomm, Giacomo Camposampiero, Aleksandar Terzic +3
We analyze the capabilities of Transformer language models in learning compositional discrete tasks. To this end, we evaluate training LLaMA models and prompting GPT-4 and Gemini o…
Zero-shot Classification using Hyperdimensional Computing
Samuele Ruffino, Geethan Karunaratne, Michael Hersche +3
Classification based on Zero-shot Learning (ZSL) is the ability of a model to classify inputs into novel classes on which the model has not previously seen any training examples. P…