4 papers
Overlapping Schwarz Attention: Hierarchical Attention via Domain Decomposition
Stephan Köhler, Oliver Rheinbach
We propose a hierarchical attention mechanism based on two-level overlapping Schwarz domain decomposition. The method is motivated by domain decomposition methods in partial differ…
How Token Influence Decays with Distance: A Green-Function View of Trained Language Models
Matthias Brändel, Stephan Köhler, Oliver Rheinbach
We study how the next-token prediction of an autoregressive Transformer language model changes under small perturbations of earlier input token embeddings. Motivated by operator le…
Combining Nonlinear FETI-DP Methods and Quasi-Newton Methods using an SQP Approach
Stephan Köhler, Oliver Rheinbach
The combination of nonlinear FETI-DP (Dual Primal Finite Element Tearing and Interconnecting) and Quasi-Newton methods using a sequential quadratic programming (SQP) approach is co…
Monolithic Multi-level Overlapping Schwarz Solvers for Fluid Problems
Stephan Köhler, Oliver Rheinbach
Additive overlapping Schwarz Methods are iterative methods of the domain decomposition type for the solution of partial differential equations. Numerical and parallel scalability o…