62 citations · 226 across the 45 of their papers we have counts for
8 papers · 1 filter
Measuring Agents in Production
Melissa Z. Pan, Negar Arabzadeh, Riccardo Cogo +22
LLM-based agents already operate in production across many industries, yet we lack an understanding of what technical methods make deployments successful. We present the first syst…
QueryGym: A Toolkit for Reproducible LLM-Based Query Reformulation
Amin Bigdeli, Radin Hamidi Rad, Mert Incesu +3
We present QueryGym, a lightweight, extensible Python toolkit that supports large language model (LLM)-based query reformulation. This is an important tool development since recent…
Adversarial Attacks against Neural Ranking Models via In-Context Learning
Amin Bigdeli, Negar Arabzadeh, Ebrahim Bagheri +1
While neural ranking models (NRMs) have shown high effectiveness, they remain susceptible to adversarial manipulation. In this work, we introduce Few-Shot Adversarial Prompting (FS…
DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesis
Liana Patel, Negar Arabzadeh, Harshit Gupta +4
The ability to research and synthesize knowledge is central to human expertise and progress. A new class of AI systems--designed for generative research synthesis--aims to automate…
Benchmarking LLM-based Relevance Judgment Methods
Negar Arabzadeh, Charles L. A. Clarke
Large Language Models (LLMs) are increasingly deployed in both academic and industry settings to automate the evaluation of information seeking systems, particularly by generating…
A Human-AI Comparative Analysis of Prompt Sensitivity in LLM-Based Relevance Judgment
Negar Arabzadeh, Charles L. A . Clarke
Large Language Models (LLMs) are increasingly used to automate relevance judgments for information retrieval (IR) tasks, often demonstrating agreement with human labels that approa…
exHarmony: Authorship and Citations for Benchmarking the Reviewer Assignment Problem
Sajad Ebrahimi, Sara Salamat, Negar Arabzadeh +2
The peer review process is crucial for ensuring the quality and reliability of scholarly work, yet assigning suitable reviewers remains a significant challenge. Traditional manual…
Benchmarking Prompt Sensitivity in Large Language Models
Amirhossein Razavi, Mina Soltangheis, Negar Arabzadeh +3
Large language Models (LLMs) are highly sensitive to variations in prompt formulation, which can significantly impact their ability to generate accurate responses. In this paper, w…