4 citations · 6 across the 9 of their papers we have counts for
9 papers
Judge, Retrieve, or Abstain: Uncertainty-Guarded LLM Judging with Provable Risk Guarantees
Sher Badshah, Ali Emami, Hassan Sajjad
Using LLMs as judges has become standard practice for evaluating model outputs at scale. This is particularly common for subjective, open-ended tasks such as assessing helpfulness…
SCOPE: Selective Conformal Optimized Pairwise LLM Judging
Sher Badshah, Ali Emami, Hassan Sajjad
Large language models (LLMs) are increasingly used as scalable judges in pairwise evaluation, but they remain prone to miscalibration and biases. We propose \textsc{Scope} (Selecti…
Interpreting the Effects of Quantization on LLMs
Manpreet Singh, Hassan Sajjad
Quantization offers a practical solution to deploy LLMs in resource-constraint environments. However, its impact on internal representations remains understudied, raising questions…
Cross-Layer Discrete Concept Discovery for Interpreting Language Models
Ankur Garg, Xuemin Yu, Hassan Sajjad +1
Interpreting language models remains challenging due to the existence of residual stream, which linearly mixes and duplicates features across adjacent layers, causing single-layer…
SAGE: A Search-AuGmented Evaluation of Large Language Models on Free-Form QA
Sher Badshah, Ali Emami, Hassan Sajjad
As Large Language Models (LLMs) become increasingly used for question-answering (QA), relying on static, pre-annotated references for evaluation poses significant challenges in cos…
CLEV: LLM-Based Evaluation Through Lightweight Efficient Voting for Free-Form Question-Answering
Sher Badshah, Moamen Moustafa, Hassan Sajjad
Evaluating free-form Question Answering (QA) remains a challenge due to its diverse and open-ended nature. Traditional automatic metrics fail to capture semantic equivalence or acc…
Which Words Matter Most in Zero-Shot Prompts?
Nikta Gohari Sadr, Sangmitra Madhusudan, Hassan Sajjad +1
While zero-shot instructional prompts like "Let's think step-by-step" have revolutionized Large Language Model performance, a fundamental question remains unanswered: which specifi…
Reference-Guided Verdict: LLMs-as-Judges in Automatic Evaluation of Free-Form QA
Sher Badshah, Hassan Sajjad
The emergence of Large Language Models (LLMs) as chat assistants capable of generating human-like conversations has amplified the need for robust evaluation methods, particularly f…
Quantifying the Capabilities of LLMs across Scale and Precision
Sher Badshah, Hassan Sajjad
Scale is often attributed as one of the factors that cause an increase in the performance of LLMs, resulting in models with billion and trillion parameters. One of the limitations…