5 papers · 1 filter
Beneath the Surface: Investigating LLMs' Capabilities for Communicating with Subtext
Kabir Ahuja, Yuxuan Li, Andrew Kyle Lampinen
Human communication is fundamentally creative, and often makes use of subtext -- implied meaning that goes beyond the literal content of the text. Here, we systematically study whe…
Finding Flawed Fictions: Evaluating Complex Reasoning in Language Models via Plot Hole Detection
Kabir Ahuja, Melanie Sclar, Yulia Tsvetkov
Stories are a fundamental aspect of human experience. Engaging deeply with stories and spotting plot holes -- inconsistencies in a storyline that break the internal logic or rules…
Learning Syntax Without Planting Trees: Understanding Hierarchical Generalization in Transformers
Kabir Ahuja, Vidhisha Balachandran, Madhur Panwar +4
Transformers trained on natural language data have been shown to learn its hierarchical structure and generalize to sentences with unseen syntactic structures without explicitly en…
Bridging the Language Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs
Somnath Kumar, Vaibhav Balloli, Mercy Ranjit +5
Large language models (LLMs) have revolutionized various domains but still struggle with non-Latin scripts and low-resource languages. This paper addresses the critical challenge o…
DIALECTBENCH: A NLP Benchmark for Dialects, Varieties, and Closely-Related Languages
Fahim Faisal, Orevaoghene Ahia, Aarohi Srivastava +4
Language technologies should be judged on their usefulness in real-world use cases. An often overlooked aspect in natural language processing (NLP) research and evaluation is langu…