26 citations · 75 across the 15 of their papers we have counts for
16 papers
Cross-Examination Framework: A Task-Agnostic Diagnostic for Information Fidelity in Text-to-Text Generation
Tathagata Raha, Clement Christophe, Nada Saadi +4
Traditional metrics like BLEU and BERTScore fail to capture semantic fidelity in generative text-to-text tasks. We adapt the Cross-Examination Framework (CEF) for a reference-free,…
Overalignment in Frontier LLMs: An Empirical Study of Sycophantic Behaviour in Healthcare
Clément Christophe, Wadood Mohammed Abdul, Prateek Munjal +3
As LLMs are increasingly integrated into clinical workflows, their tendency for sycophancy, prioritizing user agreement over factual accuracy, poses significant risks to patient sa…
Building Trust in Clinical LLMs: Bias Analysis and Dataset Transparency
Svetlana Maslenkova, Clement Christophe, Marco AF Pimentel +7
Large language models offer transformative potential for healthcare, yet their responsible and equitable development depends critically on a deeper understanding of how training da…
Bridging Language Barriers in Healthcare: A Study on Arabic LLMs
Nada Saadi, Tathagata Raha, Clément Christophe +3
This paper investigates the challenges of developing large language models (LLMs) proficient in both multilingual understanding and medical knowledge. We demonstrate that simply tr…
Named Clinical Entity Recognition Benchmark
Wadood M Abdul, Marco AF Pimentel, Muhammad Umar Salman +6
This technical report introduces a Named Clinical Entity Recognition Benchmark for evaluating language models in healthcare, addressing the crucial natural language processing (NLP…
Beyond Fine-tuning: Unleashing the Potential of Continuous Pretraining for Clinical LLMs
Clément Christophe, Tathagata Raha, Svetlana Maslenkova +4
Large Language Models (LLMs) have demonstrated significant potential in transforming clinical applications. In this study, we investigate the efficacy of four techniques in adaptin…
MEDIC: Comprehensive Evaluation of Leading Indicators for LLM Safety and Utility in Clinical Applications
Praveenkumar Kanithi, Clément Christophe, Marco AF Pimentel +8
While Large Language Models (LLMs) achieve superhuman performance on standardized medical licensing exams, these static benchmarks have become saturated and increasingly disconnect…
Med42-v2: A Suite of Clinical LLMs
Clément Christophe, Praveen K Kanithi, Tathagata Raha +2
Med42-v2 introduces a suite of clinical large language models (LLMs) designed to address the limitations of generic models in healthcare settings. These models are built on Llama3…
Beyond Metrics: A Critical Analysis of the Variability in Large Language Model Evaluation Frameworks
Marco AF Pimentel, Clément Christophe, Tathagata Raha +3
As large language models (LLMs) continue to evolve, the need for robust and standardized evaluation benchmarks becomes paramount. Evaluating the performance of these models is a co…
iREL at SemEval-2024 Task 9: Improving Conventional Prompting Methods for Brain Teasers
Harshit Gupta, Manav Chaudhary, Tathagata Raha +2
This paper describes our approach for SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense. The BRAINTEASER task comprises multiple-choice Question Answering designe…
Med42 -- Evaluating Fine-Tuning Strategies for Medical LLMs: Full-Parameter vs. Parameter-Efficient Approaches
Clément Christophe, Praveen K Kanithi, Prateek Munjal +13
This study presents a comprehensive analysis and comparison of two predominant fine-tuning methodologies - full-parameter fine-tuning and parameter-efficient tuning - within the co…
Neural models for Factual Inconsistency Classification with Explanations
Tathagata Raha, Mukund Choudhary, Abhinav Menon +4
Factual consistency is one of the most important requirements when editing high quality documents. It is extremely important for automatic text generation systems like summarizatio…
Identifying COVID-19 Fake News in Social Media
Tathagata Raha, Vijayasaradhi Indurthi, Aayush Upadhyaya +4
The evolution of social media platforms have empowered everyone to access information easily. Social media users can easily share information with the rest of the world. This may s…
Task Adaptive Pretraining of Transformers for Hostility Detection
Tathagata Raha, Sayar Ghosh Roy, Ujwal Narayan +2
Identifying adverse and hostile content on the web and more particularly, on social media, has become a problem of paramount interest in recent years. With their ever increasing po…
Leveraging Multilingual Transformers for Hate Speech Detection
Sayar Ghosh Roy, Ujwal Narayan, Tathagata Raha +2
Detecting and classifying instances of hate in social media text has been a problem of interest in Natural Language Processing in the recent years. Our work leverages state of the…
Development of POS tagger for English-Bengali Code-Mixed data
Tathagata Raha, Sainik Kumar Mahata, Dipankar Das +1
Code-mixed texts are widespread nowadays due to the advent of social media. Since these texts combine two languages to formulate a sentence, it gives rise to various research probl…