Knowing When to Stop: Efficient Context Processing via Latent Sufficiency Signals

Roy Xie, Junlin Wang, Paul Rosu +4

Large language models (LLMs) process entire input contexts indiscriminately, which is inefficient when the information required to answer a query is localized within the context. W…