The Belief State Transformer

Edward S. Hu, Kwangjun Ahn, Qinghua Liu +9

We introduce the "Belief State Transformer", a next-token predictor that takes both a prefix and suffix as inputs, with a novel objective of predicting both the next token for the…