5 papers
Length Generalization for Transformers via Compression
Georg Zetzsche, Hongjian Jiang, Andy Yang +4
Recent advancements in transformer length generalization theory enable us to reliably predict when a transformer can learn to solve a task. In particular, the C-RASP hypothesis (a…
The Complexity of Verifying Feedforward Neural Networks in Quantised Settings
Eric Alsmann, Martin Lange, Marco Sälzer
We investigate the computational complexity of neural network verification in quantised settings. We distinguish three classes of Feedforward Neural Networks (FNNs): rational FNNs…
The Polynomial Counting Capabilities of Message Passing Neural Networks
Marco Sälzer, Pascal Bergsträßer, Anthony W. Lin
The counting power of Message Passing Neural Networks (MPNN) has been the subject of many recent papers, showing that they can express logic that involves counting up to a threshol…
Transformer Encoder Satisfiability: Complexity and Impact on Formal Reasoning
Marco Sälzer, Eric Alsmann, Martin Lange
We analyse the complexity of the satisfiability problem, or similarly feasibility problem, (trSAT) for transformer encoders (TE), which naturally occurs in formal verification or i…
A Logic for Reasoning About Aggregate-Combine Graph Neural Networks
Pierre Nunn, Marco Sälzer, François Schwarzentruber +1
We propose a modal logic in which counting modalities appear in linear inequalities. We show that each formula can be transformed into an equivalent graph neural network (GNN). We…