Fourier Head: Helping Large Language Models Learn Complex Probability Distributions

Nate Gillman, Daksh Aggarwal, Michael Freeman +2

As the quality of large language models has improved, there has been increased interest in using them to model non-linguistic tokens. For example, the Decision Transformer recasts…