An Evolved Universal Transformer Memory

Edoardo Cetin, Qi Sun, Tianyu Zhao +1

Prior methods propose to offset the escalating costs of modern foundation models by dropping specific parts of their contexts with hand-designed rules, while attempting to preserve…