8 papers
The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge
Youbing Yin, Jingyu Wu, Anoop Kumar +3
LLM-as-a-Judge systems are increasingly employed for automated evaluation, yet remain susceptible to systematic biases that undermine their reliability. We demonstrate that the jud…
Probing Ambiguity: Adversarial Intent Resolution for Robust Text-to-SQL
Youbing Yin, Xujun Peng, Muhammad S. Uddin +3
Text-to-SQL systems typically assume a one-to-one mapping between natural-language queries and correct SQL translations. In practice, however, many queries are inherently ambiguous…
CAGE: Coherence-Aware Graph Encoding for Retrieval-Augmented Generation
Tong Qi, Jingyu Wu, Youbing Yin +3
Traditional Retrieval-Augmented Generation (RAG) systems score each passage independently against the query, assembling context sets that may be individually relevant yet collectiv…
SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigation
Chenyang Zhu, Jiayu Yao, Kushal Chawla +10
As autonomous agents tackle increasingly complex multi-step, multi-agent tasks, their execution trajectories have scaled beyond the constraints of even the largest context windows.…
EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectors
Amin Banayeeanzade, Qingchuan Yang, Deqing Fu +6
High-quality data is essential for modern machine learning, yet many valuable corpora are sensitive and cannot be freely shared. Synthetic data offers a practical substitute for do…
DF-RAG: Query-Aware Diversity for Retrieval-Augmented Generation
Saadat Hasan Khan, Spencer Hong, Jingyu Wu +4
Retrieval-augmented generation (RAG) is a common technique for grounding language model outputs in domain-specific information. However, RAG is often challenged by reasoning-intens…
Lessons from the Field: An Adaptable Lifecycle Approach to Applied Dialogue Summarization
Kushal Chawla, Chenyang Zhu, Pengshan Cai +9
Summarization of multi-party dialogues is a critical capability in industry, enhancing knowledge transfer and operational effectiveness across many domains. However, automatically…
RAFFLES: Reasoning-based Attribution of Faults for LLM Systems
Chenyang Zhu, Spencer Hong, Jingyu Wu +6
The advent of complex, interconnected long-horizon LLM systems has made it incredibly tricky to identify where and when these systems break down. Evaluation capabilities that curre…