5 papers
Intelligence Requires Grounding But Not Embodiment
Marcus Ma, Shrikanth Narayanan
Recent advances in LLMs have reignited scientific debate over whether embodiment is necessary for intelligence. We present the argument that intelligence requires grounding, a phen…
Authors Should Label Their Own Documents
Marcus Ma, Cole Johnson, Nolan Bridges +3
Third-party annotation is the status quo for labeling text, but egocentric information such as sentiment and belief can at best only be approximated by a third-person proxy. We int…
Semantic F1 Scores: Fair Evaluation Under Fuzzy Class Boundaries
Georgios Chochlakis, Jackson Trager, Vedant Jhaveri +3
We propose Semantic F1 Scores, novel evaluation metrics for subjective or fuzzy multi-label classification that quantify semantic relatedness between predicted and gold labels. Unl…
Large Language Models Do Multi-Label Classification Differently
Marcus Ma, Georgios Chochlakis, Niyantha Maruthu Pandiyan +2
Multi-label classification is prevalent in real-world settings, but the behavior of Large Language Models (LLMs) in this setting is understudied. We investigate how autoregressive…
Humans Hallucinate Too: Language Models Identify and Correct Subjective Annotation Errors With Label-in-a-Haystack Prompts
Georgios Chochlakis, Peter Wu, Arjun Bedi +3
Modeling complex subjective tasks in Natural Language Processing, such as recognizing emotion and morality, is considerably challenging due to significant variation in human annota…