6 papers · 1 filter
Automated Benchmark Auditing for AI Agents and Large Language Models
Junlin Wang, Federico Bianchi, Shang Zhu +4
Modern AI benchmarks operate at a complexity that outpaces traditional verification methods. Tasks authored by domain experts often contain implicit assumptions, incomplete environ…
When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework
Zhen Xu, Shang Zhu, Jue Wang +5
We investigate the challenge of applying Large Language Models (LLMs) to long texts. We propose a theoretical framework that distinguishes the failure modes of long context tasks i…
Knowing When to Stop: Efficient Context Processing via Latent Sufficiency Signals
Roy Xie, Junlin Wang, Paul Rosu +4
Large language models (LLMs) process entire input contexts indiscriminately, which is inefficient when the information required to answer a query is localized within the context. W…
SIRAG: Towards Stable and Interpretable RAG with A Process-Supervised Multi-Agent Framework
Junlin Wang, Zehao Wu, Shaowei Lu +2
Retrieval-Augmented Generation (RAG) enables large language models (LLMs) to access external knowledge sources, but the effectiveness of RAG relies on the coordination between the…
ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods
Roy Xie, Junlin Wang, Ruomin Huang +5
The rapid scaling of large language models (LLMs) has raised concerns about the transparency and fair use of the data used in their pretraining. Detecting such content is challengi…
Improving Model Alignment Through Collective Intelligence of Open-Source LLMS
Junlin Wang, Roy Xie, Shang Zhu +6
Building helpful and harmless large language models (LLMs) requires effective model alignment approach based on human instructions and feedback, which necessitates high-quality hum…