Showing 2024Show all
3 papers · 1 filter
cs.CL2024
Towards Understanding the Robustness of LLM-based Evaluations under Perturbations
Manav Chaudhary, Harshit Gupta, Savita Bhat +1
Traditional evaluation metrics like BLEU and ROUGE fall short when capturing the nuanced qualities of generated text, particularly when there is no single ground truth. In this pap…
cs.CL2024
iREL at SemEval-2024 Task 9: Improving Conventional Prompting Methods for Brain Teasers
Harshit Gupta, Manav Chaudhary, Tathagata Raha +2
This paper describes our approach for SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense. The BRAINTEASER task comprises multiple-choice Question Answering designe…
cs.CL2024
BrainStorm @ iREL at #SMM4H 2024: Leveraging Translation and Topical Embeddings for Annotation Detection in Tweets
Manav Chaudhary, Harshit Gupta, Vasudeva Varma
The proliferation of LLMs in various NLP tasks has sparked debates regarding their reliability, particularly in annotation tasks where biases and hallucinations may arise. In this…