Increasing transformer token length with a Maximum Entropy Principle Method

R. I. Cukier

Transformers suffer from the computational overhead of their quadratic dependence on the length of sequences processed. We present three methods, all adding an intermediate step be…