4 papers · 1 filter
Rating the Pitch, Not the Product: User Evaluations of LLMs Reflect Expectations More Than Performance
Robert Morabito, Tyler McDonald, Charitra Viswanath +4
Imagine two users interact with the same LLM. One has been told it is the cutting-edge flagship model; the other, an older, weaker model. They walk away with markedly different rat…
Fine-Tuned LLMs are "Time Capsules" for Tracking Societal Bias Through Books
Sangmitra Madhusudan, Robert Morabito, Skye Reid +2
Books, while often rich in cultural insights, can also mirror societal biases of their eras - biases that Large Language Models (LLMs) may learn and perpetuate during training. We…
STOP! Benchmarking Large Language Models with Sensitivity Testing on Offensive Progressions
Robert Morabito, Sangmitra Madhusudan, Tyler McDonald +1
Mitigating explicit and implicit biases in Large Language Models (LLMs) has become a critical focus in the field of natural language processing. However, many current methodologies…
Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models
Abhishek Kumar, Robert Morabito, Sanzhar Umbet +2
As the use of Large Language Models (LLMs) becomes more widespread, understanding their self-evaluation of confidence in generated responses becomes increasingly important as it is…