7 papers
Uncovering the Computational Ingredients of Human-Like Representations in LLMs
Zach Studdiford, Timothy T. Rogers, Kushin Mukherjee +1
The ability to translate diverse patterns of inputs into structured patterns of behavior has been thought to rest on both humans' and machines' ability to learn robust representati…
Probing LLM World Models: Enhancing Guesstimation with Wisdom of Crowds Decoding
Yun-Shiuan Chuang, Sameer Narendran, Nikunj Harlalka +5
Guesstimation -- the task of making approximate quantitative estimates about objects or events -- is a common real-world skill, yet remains underexplored in large language model (L…
Which LLMs Get the Joke? Probing Non-STEM Reasoning Abilities with HumorBench
Reuben Narad, Siddharth Suresh, Jiayi Chen +5
We present HumorBench, a benchmark designed to evaluate large language models' (LLMs) ability to reason about and explain sophisticated humor in cartoon captions. As reasoning mode…
Evaluating Steering Techniques using Human Similarity Judgments
Zach Studdiford, Timothy T. Rogers, Siddharth Suresh +1
Current evaluations of Large Language Model (LLM) steering techniques focus on task-specific performance, overlooking how well steered representations align with human cognition. U…
AI-enhanced semantic feature norms for 786 concepts
Siddharth Suresh, Kushin Mukherjee, Tyler Giallanza +4
Semantic feature norms have been foundational in the study of human conceptual knowledge, yet traditional methods face trade-offs between concept/feature coverage and verifiability…
Bridging the Creativity Understanding Gap: Small-Scale Human Alignment Enables Expert-Level Humor Ranking in LLMs
Kuan Lok Zhou, Jiayi Chen, Siddharth Suresh +6
Large Language Models (LLMs) have shown significant limitations in understanding creative content, as demonstrated by Hessel et al. (2023)'s influential work on the New Yorker Cart…