67 citations · 71 across the 7 of their papers we have counts for
11 papers
Length Generalization Needs Proper Regularization
Pavlo Vasylenko, Matthias Lindemann, André F. T. Martins +1
Length generalization is the ability of sequential models to perform well on context lengths unseen during training. In this work, we show that the challenge of achieving length ge…
MultiSynt/MT: Trillion-Token Multi-Parallel Pre-Training Data Translated Across 36 Languages
Maximilian Idahl, Jörg Tiedemann, Sampo Pyysalo +19
Open web-scale pre-training corpora remain concentrated in English, limiting multilingual LLM development. We introduce MultiSynt/MT, an open synthetic parallel corpus with approxi…
Strengthening Structural Inductive Biases by Pre-training to Perform Syntactic Transformations
Matthias Lindemann, Alexander Koller, Ivan Titov
Models need appropriate inductive biases to effectively learn from small amounts of data and generalize systematically outside of the training distribution. While Transformers are…
Cache & Distil: Optimising API Calls to Large Language Models
Guillem Ramírez, Matthias Lindemann, Alexandra Birch +1
Large-scale deployment of generative AI tools often depends on costly API calls to a Large Language Model (LLM) to fulfil user queries. To curtail the frequency of these calls, one…
SIP: Injecting a Structural Inductive Bias into a Seq2Seq Model by Simulation
Matthias Lindemann, Alexander Koller, Ivan Titov
Strong inductive biases enable learning from little data and help generalization outside of the training distribution. Popular neural architectures such as Transformers lack strong…
Compositional Generalization without Trees using Multiset Tagging and Latent Permutations
Matthias Lindemann, Alexander Koller, Ivan Titov
Seq2seq models have been shown to struggle with compositional generalization in semantic parsing, i.e. generalizing to unseen compositions of phenomena that the model handles corre…
Compositional Generalisation with Structured Reordering and Fertility Layers
Matthias Lindemann, Alexander Koller, Ivan Titov
Seq2seq models have been shown to struggle with compositional generalisation, i.e. generalising to new and potentially more complex structures than seen during training. Taking ins…
Fast semantic parsing with well-typedness guarantees
Matthias Lindemann, Jonas Groschwitz, Alexander Koller
AM dependency parsing is a linguistically principled method for neural semantic parsing with high accuracy across multiple graphbanks. It relies on a type system that models semant…
Normalizing Compositional Structures Across Graphbanks
Lucia Donatelli, Jonas Groschwitz, Alexander Koller +2
The emergence of a variety of graph-based meaning representations (MRs) has sparked an important conversation about how to adequately represent semantic structure. These MRs exhibi…
Compositional Semantic Parsing Across Graphbanks
Matthias Lindemann, Jonas Groschwitz, Alexander Koller
Most semantic parsers that map sentences to graph-based meaning representations are hand-designed for specific graphbanks. We present a compositional neural semantic parser which a…
AMR Dependency Parsing with a Typed Semantic Algebra
Jonas Groschwitz, Matthias Lindemann, Meaghan Fowlie +2
We present a semantic parser for Abstract Meaning Representations which learns to parse strings into tree representations of the compositional structure of an AMR graph. This allow…