Enhancing the Transformer with Explicit Relational Encoding for Math Problem Solving

Imanol Schlag, Paul Smolensky, Roland Fernandez +3

We incorporate Tensor-Product Representations within the Transformer in order to better support the explicit representation of relation structure. Our Tensor-Product Transformer (T…