6 papers
DiaLLM: An Investigation into the Robustness-Generation Gap in English Dialect Adaptation
Jordan Painter, Dipankar Srirag, Adarsh Kappiyath +3
Large language models increasingly \emph{understand} dialectal English, yet still \emph{produce} only standard, US-leaning English, leaving dialectal generation, the harder half of…
Nek Minit: Harnessing Pragmatic Metacognitive Prompting for Explainable Sarcasm Detection of Australian and Indian English
Ishmanbir Singh, Dipankar Srirag, Aditya Joshi
Sarcasm is a challenge to sentiment analysis because of the incongruity between stated and implied sentiment. The challenge is exacerbated when the implication may be relevant to a…
BESSTIE: A Benchmark for Sentiment and Sarcasm Classification for Varieties of English
Dipankar Srirag, Aditya Joshi, Jordan Painter +1
Despite large language models (LLMs) being known to exhibit bias against non-standard language varieties, there are no known labelled datasets for sentiment analysis of English. To…
Predicting the Target Word of Game-playing Conversations using a Low-Rank Dialect Adapter for Decoder Models
Dipankar Srirag, Aditya Joshi, Jacob Eisenstein
Dialect adapters that improve the performance of LLMs for NLU tasks on certain sociolects/dialects/national varieties ('dialects' for the sake of brevity) have been reported for en…
Evaluating Dialect Robustness of Language Models via Conversation Understanding
Dipankar Srirag, Nihar Ranjan Sahoo, Aditya Joshi
With an evergrowing number of LLMs reporting superlative performance for English, their ability to perform equitably for different dialects of English (, dialect rob…
Experiences from Creating a Benchmark for Sentiment Classification for Varieties of English
Dipankar Srirag, Jordan Painter, Aditya Joshi +1
Existing benchmarks often fail to account for linguistic diversity, like language variants of English. In this paper, we share our experiences from our ongoing project of building…