4 citations · 11 across the 15 of their papers we have counts for
11 papers · 1 filter
Augmenting LLM Reasoning with Dynamic Notes Writing for Complex QA
Rishabh Maheshwary, Masoud Hashemi, Khyati Mahajan +5
Iterative RAG for multi-hop question answering faces challenges with lengthy contexts and the buildup of irrelevant information. This hinders a model's capacity to process and reas…
Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation
Israfel Salazar, Manuel Fernández Burda, Shayekh Bin Islam +42
The evaluation of vision-language models (VLMs) has mainly relied on English-language benchmarks, leaving significant gaps in both multilingual and multicultural coverage. While mu…
INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge
Angelika Romanou, Negar Foroutan, Anna Sotnikova +56
The performance differential of large language models (LLM) between languages hinders their effective deployment in many regions, inhibiting the potential economic and societal val…
M-RewardBench: Evaluating Reward Models in Multilingual Settings
Srishti Gureja, Lester James V. Miranda, Shayekh Bin Islam +7
Reward models (RMs) have driven the state-of-the-art performance of LLMs today by enabling the integration of human feedback into the language modeling process. However, RMs are pr…
M2Lingual: Enhancing Multilingual, Multi-Turn Instruction Alignment in Large Language Models
Rishabh Maheshwary, Vikas Yadav, Hoang Nguyen +2
Instruction finetuning (IFT) is critical for aligning Large Language Models (LLMs) to follow instructions. While many effective IFT datasets have been introduced recently, they pre…
Curry-DPO: Enhancing Alignment using Curriculum Learning & Ranked Preferences
Pulkit Pattnaik, Rishabh Maheshwary, Kelechi Ogueji +2
Direct Preference Optimization (DPO) is an effective technique that leverages pairwise preference data (usually one chosen and rejected response pair per user prompt) to align LLMs…
Practice Makes a Solver Perfect: Data Augmentation for Math Word Problem Solvers
Vivek Kumar, Rishabh Maheshwary, Vikram Pudi
Existing Math Word Problem (MWP) solvers have achieved high accuracy on benchmark datasets. However, prior works have shown that such solvers do not generalize well and rely on sup…
Adversarial Examples for Evaluating Math Word Problem Solvers
Vivek Kumar, Rishabh Maheshwary, Vikram Pudi
Standard accuracy metrics have shown that Math Word Problem (MWP) solvers have achieved high performance on benchmark datasets. However, the extent to which existing MWP solvers tr…
A Strong Baseline for Query Efficient Attacks in a Black Box Setting
Rishabh Maheshwary, Saket Maheshwary, Vikram Pudi
Existing black box search methods have achieved high success rate in generating adversarial attacks against NLP models. However, such search methods are inefficient as they do not…
A Context Aware Approach for Generating Natural Language Attacks
Rishabh Maheshwary, Saket Maheshwary, Vikram Pudi
We study an important task of attacking natural language processing models in a black box setting. We propose an attack strategy that crafts semantically similar adversarial exampl…
Generating Natural Language Attacks in a Hard Label Black Box Setting
Rishabh Maheshwary, Saket Maheshwary, Vikram Pudi
We study an important and challenging task of attacking natural language processing models in a hard label black box setting. We propose a decision-based attack strategy that craft…