3 citations · 5 across the 18 of their papers we have counts for
20 papers
The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge
Youbing Yin, Jingyu Wu, Anoop Kumar +3
LLM-as-a-Judge systems are increasingly employed for automated evaluation, yet remain susceptible to systematic biases that undermine their reliability. We demonstrate that the jud…
Probing Ambiguity: Adversarial Intent Resolution for Robust Text-to-SQL
Youbing Yin, Xujun Peng, Muhammad S. Uddin +3
Text-to-SQL systems typically assume a one-to-one mapping between natural-language queries and correct SQL translations. In practice, however, many queries are inherently ambiguous…
CAGE: Coherence-Aware Graph Encoding for Retrieval-Augmented Generation
Tong Qi, Jingyu Wu, Youbing Yin +3
Traditional Retrieval-Augmented Generation (RAG) systems score each passage independently against the query, assembling context sets that may be individually relevant yet collectiv…
SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigation
Chenyang Zhu, Jiayu Yao, Kushal Chawla +10
As autonomous agents tackle increasingly complex multi-step, multi-agent tasks, their execution trajectories have scaled beyond the constraints of even the largest context windows.…
Alignment-Weighted DPO: A principled reasoning approach to improve safety alignment
Mengxuan Hu, Vivek V. Datla, Anoop Kumar +4
Recent advances in alignment techniques such as Supervised Fine-Tuning (SFT), Reinforcement Learning from Human Feedback (RLHF), and Direct Preference Optimization (DPO) have impro…
DF-RAG: Query-Aware Diversity for Retrieval-Augmented Generation
Saadat Hasan Khan, Spencer Hong, Jingyu Wu +4
Retrieval-augmented generation (RAG) is a common technique for grounding language model outputs in domain-specific information. However, RAG is often challenged by reasoning-intens…
LLM Optimization Unlocks Real-Time Pairwise Reranking
Jingyu Wu, Aditya Shrivastava, Jing Zhu +3
Efficiently reranking documents retrieved from information retrieval (IR) pipelines to enhance overall quality of Retrieval-Augmented Generation (RAG) system remains an important y…
Uncertainty as Feature Gaps: Epistemic Uncertainty Quantification of LLMs in Contextual Question-Answering
Yavuz Bakman, Sungmin Kang, Zhiqi Huang +8
Uncertainty Quantification (UQ) research has primarily focused on closed-book factual question answering (QA), while contextual QA remains unexplored, despite its importance in rea…
Readability Reconsidered: A Cross-Dataset Analysis of Reference-Free Metrics
Catarina G Belem, Parker Glenn, Alfy Samuel +2
Automatic readability assessment plays a key role in ensuring effective and accessible written communication. Despite significant progress, the field is hindered by inconsistent de…
A Comparison of Independent and Joint Fine-tuning Strategies for Retrieval-Augmented Generation
Neal Gregory Lawton, Alfy Samuel, Anoop Kumar +1
A Comparison of Independent and Joint Fine-tuning Strategies for Retrieval-Augmented Generation Download PDF Neal Gregory Lawton, Alfy Samuel, Anoop Kumar, Daben Liu Published: 20…
Harmonizing Diverse Models: A Layer-wise Merging Strategy for Consistent Generation
Xujun Peng, Anoop Kumar, Jingyu Wu +2
Retrieval-Augmented Generation (RAG) systems leverage Large Language Models (LLMs) to generate accurate and reliable responses that are grounded in retrieved context. However, LLMs…
Confidence-Based Response Abstinence: Improving LLM Trustworthiness via Activation-Based Uncertainty Estimation
Zhiqi Huang, Vivek Datla, Chenyang Zhu +4
We propose a method for confidence estimation in retrieval-augmented generation (RAG) systems that aligns closely with the correctness of large language model (LLM) outputs. Confid…
Improving Consistency in Retrieval-Augmented Systems with Group Similarity Rewards
Faisal Hamman, Chenyang Zhu, Anoop Kumar +4
RAG systems are increasingly deployed in high-stakes domains where users expect outputs to be consistent across semantically equivalent queries. However, existing systems often exh…
Play by the Type Rules: Inferring Constraints for LLM Functions in Declarative Programs
Parker Glenn, Alfy Samuel, Daben Liu
Integrating LLM powered operators in declarative query languages allows for the combination of cheap and interpretable functions with powerful, generalizable language model reasoni…
RAFFLES: Reasoning-based Attribution of Faults for LLM Systems
Chenyang Zhu, Spencer Hong, Jingyu Wu +6
The advent of complex, interconnected long-horizon LLM systems has made it incredibly tricky to identify where and when these systems break down. Evaluation capabilities that curre…
TruthTorchLM: A Comprehensive Library for Predicting Truthfulness in LLM Outputs
Duygu Nur Yaldiz, Yavuz Faruk Bakman, Sungmin Kang +9
Generative Large Language Models (LLMs)inevitably produce untruthful responses. Accurately predicting the truthfulness of these outputs is critical, especially in high-stakes setti…
FB-RAG: Improving RAG with Forward and Backward Lookup
Kushal Chawla, Alfy Samuel, Anoop Kumar +1
Traditional Retrieval-Augmented Generation (RAG) struggles with complex queries that lack strong signals to retrieve the most relevant context, forcing a trade-off between choosing…
Refusal Tokens: A Simple Way to Calibrate Refusals in Large Language Models
Neel Jain, Aditya Shrivastava, Chenyang Zhu +6
A key component of building safe and reliable language models is enabling the models to appropriately refuse to follow certain instructions or answer certain questions. We may want…
Retrieval Augmented Correction of Named Entity Speech Recognition Errors
Ernest Pusateri, Anmol Walia, Anirudh Kashi +6
In recent years, end-to-end automatic speech recognition (ASR) systems have proven themselves remarkably accurate and performant, but these systems still have a significant error r…
Predicting Stress in Remote Learning via Advanced Deep Learning Technologies
Daben Kyle Liu
COVID-19 has driven most schools to remote learning through online meeting software such as Zoom and Google Meet. Although this trend helps students continue learning without in-pe…