SIP: Injecting a Structural Inductive Bias into a Seq2Seq Model by Simulation

Matthias Lindemann, Alexander Koller, Ivan Titov

Strong inductive biases enable learning from little data and help generalization outside of the training distribution. Popular neural architectures such as Transformers lack strong…