6 papers
One Reference or Many? Rethinking Multi-Reference Training for Intermediate-Entropy Text Generation
Maitreya Prafulla Chitale, Ketaki Mangesh Shetye, Yash More +4
Natural language generation (NLG) ranges from highly constrained tasks, such as machine translation, to open-ended tasks like dialogue generation. Between them lie peer-review gene…
Graph-Guided Passage Retrieval for Author-Centric Structured Feedback
Maitreya Prafulla Chitale, Ketaki Mangesh Shetye, Harshit Gupta +3
Obtaining high-quality, pre-submission feedback is a critical bottleneck in the academic publication lifecycle for researchers. We introduce AutoRev, an automated author-centric fe…
Can Code Language Models Learn Clarification-Seeking Behaviors?
Jie JW Wu, Manav Chaudhary, Davit Abrahamyan +3
Large language models (LLMs) have demonstrated remarkable capabilities in code generation tasks. However, a gap remains between their output and the problem-solving strategies of h…
Towards Understanding the Robustness of LLM-based Evaluations under Perturbations
Manav Chaudhary, Harshit Gupta, Savita Bhat +1
Traditional evaluation metrics like BLEU and ROUGE fall short when capturing the nuanced qualities of generated text, particularly when there is no single ground truth. In this pap…
iREL at SemEval-2024 Task 9: Improving Conventional Prompting Methods for Brain Teasers
Harshit Gupta, Manav Chaudhary, Tathagata Raha +2
This paper describes our approach for SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense. The BRAINTEASER task comprises multiple-choice Question Answering designe…
BrainStorm @ iREL at #SMM4H 2024: Leveraging Translation and Topical Embeddings for Annotation Detection in Tweets
Manav Chaudhary, Harshit Gupta, Vasudeva Varma
The proliferation of LLMs in various NLP tasks has sparked debates regarding their reliability, particularly in annotation tasks where biases and hallucinations may arise. In this…