6 papers
Judging a Review by its Cover: A Reliability Analysis of LLM-based Peer Review Evaluation Metrics
Shakiba Amirshahi, Sajad Ebrahimi, Hai Son Le +2
Peer-review evaluation is increasingly being automated with LLM-as-a-judge metrics, but this creates a measurement risk. A review may receive a high score because it is fluent, org…
Route Me If You Can: A Benchmark for Query Reformulation Selection
Hai Son Le, Negar Arabzadeh, Amin Bigdeli +4
LLM-based query reformulation can improve retrieval, but no single reformulation strategy is consistently optimal across queries, domains, retrievers, or model backbones. This crea…
EviQE: Evidence Selection for LLM-Based Query Expansion
Hai Son Le, Amin Bigdeli, Shirin Seyedsalehi +2
LLM-based query expansion increasingly conditions reformulation on documents retrieved from the target corpus, yet most work focuses on how to generate expansions rather than which…
A Reproducibility Study of LLM-Based Query Reformulation
Amin Bigdeli, Radin Hamidi Rad, Hai Son Le +4
Large Language Models (LLMs) are now widely used for query reformulation and expansion in Information Retrieval, with many studies reporting substantial effectiveness gains. Howeve…
PeeriScope: A Multi-Faceted Framework for Evaluating Peer Review Quality
Sajad Ebrahimi, Soroush Sadeghian, Ali Ghorbanpour +6
The increasing scale and variability of peer review in scholarly venues has created an urgent need for systematic, interpretable, and extensible tools to assess review quality. We…
SocialHarmBench: Revealing LLM Vulnerabilities to Socially Harmful Requests
Punya Syon Pandey, Hai Son Le, Devansh Bhardwaj +2
Large language models (LLMs) are increasingly deployed in contexts where their failures can have direct sociopolitical consequences. Yet, existing safety benchmarks rarely test vul…