Let's (not) just put things in Context: Test-Time Training for Long-Context LLMs

Rachit Bansal, Aston Zhang, Rishabh Tiwari +8

Progress on training and architecture strategies has enabled LLMs with millions of tokens in context length. However, empirical evidence suggests that such long-context LLMs can co…