14 citations · 28 across the 11 of their papers we have counts for
12 papers
Protect: Towards Robust Guardrailing Stack for Trustworthy Enterprise LLM Systems
Karthik Avinash, Nikhil Pareek, Rishav Hada
The increasing deployment of Large Language Models (LLMs) across enterprise and mission-critical domains has underscored the urgent need for robust guardrailing systems that ensure…
AgentCompass: Towards Reliable Evaluation of Agentic Workflows in Production
NVJK Kartik, Garvit Sapra, Rishav Hada +1
With the growing adoption of Large Language Models (LLMs) in automating complex, multi-agent workflows, organizations face mounting risks from errors, emergent behaviors, and syste…
Akal Badi ya Bias: An Exploratory Study of Gender Bias in Hindi Language Technology
Rishav Hada, Safiya Husain, Varun Gumma +8
Existing research in measuring and mitigating gender bias predominantly centers on English, overlooking the intricate challenges posed by non-English languages and the Global South…
METAL: Towards Multilingual Meta-Evaluation
Rishav Hada, Varun Gumma, Mohamed Ahmed +2
With the rising human-like precision of Large Language Models (LLMs) in numerous tasks, their utilization in a variety of real-world applications is becoming more prevalent. Severa…
BEnQA: A Question Answering and Reasoning Benchmark for Bengali and English
Sheikh Shafayat, H M Quamran Hasan, Minhajur Rahman Chowdhury Mahim +3
In this study, we introduce BEnQA, a dataset comprising parallel Bengali and English exam questions for middle and high school levels in Bangladesh. Our dataset consists of approxi…
MunTTS: A Text-to-Speech System for Mundari
Varun Gumma, Rishav Hada, Aditya Yadavalli +4
We present MunTTS, an end-to-end text-to-speech (TTS) system specifically for Mundari, a low-resource Indian language of the Austo-Asiatic family. Our work addresses the gap in lin…
MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks
Sanchit Ahuja, Divyanshu Aggarwal, Varun Gumma +8
There has been a surge in LLM evaluation research to understand LLM capabilities and limitations. However, much of this research has been confined to English, leaving LLM building…
''Fifty Shades of Bias'': Normative Ratings of Gender Bias in GPT Generated English Text
Rishav Hada, Agrima Seth, Harshita Diddee +1
Language serves as a powerful tool for the manifestation of societal belief systems. In doing so, it also perpetuates the prevalent biases in our society. Gender bias is one of the…
Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?
Rishav Hada, Varun Gumma, Adrian de Wynter +5
Large Language Models (LLMs) excel in various Natural Language Processing (NLP) tasks, yet their evaluation, particularly in languages beyond the top , remains inadequate due t…
MEGA: Multilingual Evaluation of Generative AI
Kabir Ahuja, Harshita Diddee, Rishav Hada +9
Generative AI models have shown impressive performance on many Natural Language Processing tasks such as language understanding, reasoning, and language generation. An important qu…
Beyond Digital "Echo Chambers": The Role of Viewpoint Diversity in Political Discussion
Rishav Hada, Amir Ebrahimi Fard, Sarah Shugars +5
Increasingly taking place in online spaces, modern political conversations are typically perceived to be unproductively affirming -- siloed in so called ``echo chambers'' of exclus…
Ruddit: Norms of Offensiveness for English Reddit Comments
Rishav Hada, Sohi Sudhir, Pushkar Mishra +3
On social media platforms, hateful and offensive language negatively impact the mental well-being of users and the participation of people from diverse backgrounds. Automatic metho…