LLMs are Bayesian, In Expectation, Not in Realization

Leon Chlon, Fatima Sheaib, Zein Khamis +3

Bayesian accounts of in-context learning face a direct objection: exact posterior predictives for exchangeable data are invariant to task-preserving order, yet transformers change…