4 papers · 1 filter
BESSTIE: A Benchmark for Sentiment and Sarcasm Classification for Varieties of English
Dipankar Srirag, Aditya Joshi, Jordan Painter +1
Despite large language models (LLMs) being known to exhibit bias against non-standard language varieties, there are no known labelled datasets for sentiment analysis of English. To…
Experiences from Creating a Benchmark for Sentiment Classification for Varieties of English
Dipankar Srirag, Jordan Painter, Aditya Joshi +1
Existing benchmarks often fail to account for linguistic diversity, like language variants of English. In this paper, we share our experiences from our ongoing project of building…
Predicting the Target Word of Game-playing Conversations using a Low-Rank Dialect Adapter for Decoder Models
Dipankar Srirag, Aditya Joshi, Jacob Eisenstein
Dialect adapters that improve the performance of LLMs for NLU tasks on certain sociolects/dialects/national varieties ('dialects' for the sake of brevity) have been reported for en…
Evaluating Dialect Robustness of Language Models via Conversation Understanding
Dipankar Srirag, Nihar Ranjan Sahoo, Aditya Joshi
With an evergrowing number of LLMs reporting superlative performance for English, their ability to perform equitably for different dialects of English (, dialect rob…