6 citations · 24 across the 14 of their papers we have counts for
14 papers
SurakshaEval: An Indic Safety Benchmark for Multilingual LLMs
Debopriyo Banerjee, Kapil Rajesh Kavitha, Angana Borah +11
Existing safety evaluation datasets for large language models (LLMs) predominantly focus on English and Western contexts, often overlooking the linguistic diversity and culturally…
Evaluating Evaluation Metrics -- The Mirage of Hallucination Detection
Atharva Kulkarni, Yuan Zhang, Joel Ruben Antony Moniz +5
Hallucinations pose a significant obstacle to the reliability and widespread adoption of language models, yet their accurate measurement remains a persistent challenge. While many…
Still Not Quite There! Evaluating Large Language Models for Comorbid Mental Health Diagnosis
Amey Hengle, Atharva Kulkarni, Shantanu Patankar +4
In this study, we introduce ANGST, a novel, first-of-its kind benchmark for depression-anxiety comorbidity classification from social media posts. Unlike contemporary datasets that…
SynthDST: Synthetic Data is All You Need for Few-Shot Dialog State Tracking
Atharva Kulkarni, Bo-Hsiang Tseng, Joel Ruben Antony Moniz +3
In-context learning with Large Language Models (LLMs) has emerged as a promising avenue of research in Dialog State Tracking (DST). However, the best-performing in-context learning…
Multitask Learning Can Improve Worst-Group Outcomes
Atharva Kulkarni, Lucio Dery, Amrith Setlur +3
In order to create machine learning systems that serve a variety of users well, it is vital to not only achieve high average performance but also ensure equitable outcomes across d…
Counting the Bugs in ChatGPT's Wugs: A Multilingual Investigation into the Morphological Capabilities of a Large Language Model
Leonie Weissweiler, Valentin Hofmann, Anjali Kantharuban +10
Large language models (LLMs) have recently reached an impressive level of linguistic capability, prompting comparisons with human language skills. However, there have been relative…
Revisiting Hate Speech Benchmarks: From Data Curation to System Deployment
Atharva Kulkarni, Sarah Masud, Vikram Goyal +1
Social media is awash with hateful content, much of which is often veiled with linguistic and topical diversity. The benchmark datasets used for hate speech detection do not accoun…
SciFix: Outperforming GPT3 on Scientific Factual Error Correction
Dhananjay Ashok, Atharva Kulkarni, Hai Pham +1
Due to the prohibitively high cost of creating error correction datasets, most Factual Claim Correction methods rely on a powerful verification model to guide the correction proces…
Learning and Reasoning Multifaceted and Longitudinal Data for Poverty Estimates and Livelihood Capabilities of Lagged Regions in Rural India
Atharva Kulkarni, Raya Das, Ravi S. Srivastava +1
Poverty is a multifaceted phenomenon linked to the lack of capabilities of households to earn a sustainable livelihood, increasingly being assessed using multidimensional indicator…
Characterizing the Entities in Harmful Memes: Who is the Hero, the Villain, the Victim?
Shivam Sharma, Atharva Kulkarni, Tharun Suresh +4
Memes can sway people's opinions over social media as they combine visual and textual information in an easy-to-consume manner. Since memes instantly turn viral, it becomes crucial…
Empowering the Fact-checkers! Automatic Identification of Claim Spans on Twitter
Megha Sundriyal, Atharva Kulkarni, Vaibhav Pulastya +2
The widespread diffusion of medical and political claims in the wake of COVID-19 has led to a voluminous rise in misinformation and fake news. The current vogue is to employ manual…
When did you become so smart, oh wise one?! Sarcasm Explanation in Multi-modal Multi-party Dialogues
Shivani Kumar, Atharva Kulkarni, Md Shad Akhtar +1
Indirect speech such as sarcasm achieves a constellation of discourse goals in human communication. While the indirectness of figurative language warrants speakers to achieve certa…
PVG at WASSA 2021: A Multi-Input, Multi-Task, Transformer-Based Architecture for Empathy and Distress Prediction
Atharva Kulkarni, Sunanda Somwase, Shivam Rajput +1
Active research pertaining to the affective phenomenon of empathy and distress is invaluable for improving human-machine interaction. Predicting intensities of such complex emotion…
An Attention Ensemble Approach for Efficient Text Classification of Indian Languages
Atharva Kulkarni, Amey Hengle, Rutuja Udyawar
The recent surge of complex attention-based deep learning architectures has led to extraordinary results in various downstream NLP tasks in the English language. However, such rese…