5 papers
Judging with Personality and Confidence: A Study on Personality-Conditioned LLM Relevance Assessment
Nuo Chen, Hanpei Fang, Piaohong Wang +3
Recent studies have shown that prompting can enable large language models (LLMs) to simulate specific personality traits and produce behaviors that align with those traits. However…
Mitigating the Threshold Priming Effect in Large Language Model-Based Relevance Judgments via Personality Infusing
Nuo Chen, Hanpei Fang, Jiqun Liu +3
Recent research has explored LLMs as scalable tools for relevance labeling, but studies indicate they are susceptible to priming effects, where prior relevance judgments influence…
The Bias is in the Details: An Assessment of Cognitive Bias in LLMs
R. Alexander Knipper, Charles S. Knipper, Kaiqi Zhang +3
As Large Language Models (LLMs) are increasingly embedded in real-world decision-making processes, it becomes crucial to examine the extent to which they exhibit cognitive biases.…
AI Can Be Cognitively Biased: An Exploratory Study on Threshold Priming in LLM-Based Batch Relevance Assessment
Nuo Chen, Jiqun Liu, Xiaoyu Dong +3
Cognitive biases are systematic deviations in thinking that lead to irrational judgments and problematic decision-making, extensively studied across various fields. Recently, large…
Decoy Effect In Search Interaction: Understanding User Behavior and Measuring System Vulnerability
Nuo Chen, Jiqun Liu, Hanpei Fang +3
This study examines the decoy effect's underexplored influence on user search interactions and methods for measuring information retrieval (IR) systems' vulnerability to this effec…