52 citations · 62 across the 19 of their papers we have counts for
23 papers
Evaluating Multi-Dimensional Generalization of Large Language Models in Temporal Extraction Tasks
Fahmid Shahriar Iqbal, Ritam Dutt, Soumitra Das +2
Time and event expression extraction are fundamental temporal reasoning tasks, but the problem remains difficult due to annotation ambiguity, domain sensitivity, and unstable model…
Prompt Robustness Is Task-Dependent: Comparing Objective and Belief-Style Questions in LLM Evaluation
Sadia Kamal, Arefa Patwary, Anthony Marchiafava +2
Survey-style evaluations of large language models often treat a prompted response as a measure of a model's values or beliefs. This assumption is particularly fragile when response…
S-GRADES -- Studying Generalization of Student Response Assessments in Diverse Evaluative Settings
Tasfia Seuti, Sagnik Ray Choudhury
Evaluating student responses, from long essays to short factual answers, is a key challenge in educational NLP. Automated Essay Scoring (AES) focuses on holistic writing qualities…
LFQA-HP-1M: A Large-Scale Human Preference Dataset for Long-Form Question Answering
Rafid Ishrak Jahan, Fahmid Shahriar Iqbal, Sagnik Ray Choudhury
Long-form question answering (LFQA) demands nuanced evaluation of multi-sentence explanatory responses, yet existing metrics often fail to reflect human judgment. We present LFQA-H…
Evaluation Framework for Highlight Explanations of Context Utilisation in Language Models
Jingyi Sun, Pepa Atanasova, Sagnik Ray Choudhury +2
Context utilisation, the ability of Language Models (LMs) to incorporate relevant information from the provided context when generating responses, remains largely opaque to users,…
ClaimIQ at CheckThat! 2025: Comparing Prompted and Fine-Tuned Language Models for Verifying Numerical Claims
Anirban Saha Anik, Md Fahimul Kabir Chowdhury, Andrew Wyckoff +1
This paper presents our system for Task 3 of the CLEF 2025 CheckThat! Lab, which focuses on verifying numerical and temporal claims using retrieved evidence. We explore two complem…
A Detailed Factor Analysis for the Political Compass Test: Navigating Ideologies of Large Language Models
Sadia Kamal, Lalu Prasad Yadav Prakash, S M Rafiuddin +3
The Political Compass Test (PCT) and similar surveys are commonly used to assess political bias in auto-regressive LLMs. Our rigorous statistical experiments show that while change…
Towards Automated Situation Awareness: A RAG-Based Framework for Peacebuilding Reports
Poli A. Nemkova, Suleyman O. Polat, Rafid I. Jahan +4
Timely and accurate situation awareness is vital for decision-making in humanitarian response, conflict monitoring, and early warning and early action. However, the manual analysis…
Do Large Language Models Know Conflict? Investigating Parametric vs. Non-Parametric Knowledge of LLMs for Conflict Forecasting
Apollinaire Poli Nemkova, Sarath Chandra Lingareddy, Sagnik Ray Choudhury +1
Large Language Models (LLMs) have shown impressive performance across natural language tasks, but their ability to forecast violent conflict remains underexplored. We investigate w…
NLP for Social Good: A Survey and Outlook of Challenges, Opportunities, and Responsible Deployment
Antonia Karamolegkou, Angana Borah, Eunjung Cho +30
Natural language processing (NLP) now shapes many aspects of our world, yet its potential for positive social impact is underexplored. This paper surveys work in ``NLP for Social G…
BAGELS: Benchmarking the Automated Generation and Extraction of Limitations from Scholarly Text
Ibrahim Al Azher, Miftahul Jannat Mokarrama, Zhishuai Guo +2
In scientific research, ``limitations'' refer to the shortcomings, constraints, or weaknesses of a study. A transparent reporting of such limitations can enhance the quality and re…
FutureGen: A RAG-based Approach to Generate the Future Work of Scientific Article
Ibrahim Al Azher, Miftahul Jannat Mokarrama, Zhishuai Guo +2
The Future Work section of a scientific article outlines potential research directions by identifying gaps and limitations of a current study. This section serves as a valuable res…
Navigating the Landscape of Reproducible Research: A Predictive Modeling Approach
Akhil Pandey Akella, Sagnik Ray Choudhury, David Koop +1
The reproducibility of scientific articles is central to the advancement of science. Despite this importance, evaluating reproducibility remains challenging due to the scarcity of…
Implications of Annotation Artifacts in Edge Probing Test Datasets
Sagnik Ray Choudhury, Jushaan Kalra
Edge probing tests are classification tasks that test for grammatical knowledge encoded in token representations coming from contextual encoders such as large language models (LLMs…
Explaining Interactions Between Text Spans
Sagnik Ray Choudhury, Pepa Atanasova, Isabelle Augenstein
Reasoning over spans of tokens from different parts of the input is essential for natural language understanding (NLU) tasks such as fact-checking (FC), machine reading comprehensi…
Machine Reading, Fast and Slow: When Do Models "Understand" Language?
Sagnik Ray Choudhury, Anna Rogers, Isabelle Augenstein
Two of the most fundamental challenges in Natural Language Understanding (NLU) at present are: (a) how to establish whether deep learning-based models score highly on NLU benchmark…
Can Edge Probing Tasks Reveal Linguistic Knowledge in QA Models?
Sagnik Ray Choudhury, Nikita Bhutani, Isabelle Augenstein
There have been many efforts to try to understand what grammatical knowledge (e.g., ability to understand the part of speech of a token) is encoded in large pre-trained language mo…
Intent Features for Rich Natural Language Understanding
Brian Lester, Sagnik Ray Choudhury, Rashmi Prasad +1
Complex natural language understanding modules in dialog systems have a richer understanding of user utterances, and thus are critical in providing a better user experience. Howeve…
Constrained Decoding for Computationally Efficient Named Entity Recognition Taggers
Brian Lester, Daniel Pressel, Amy Hemmeter +2
Current state-of-the-art models for named entity recognition (NER) are neural models with a conditional random field (CRF) as the final layer. Entities are represented as per-token…
Multiple Word Embeddings for Increased Diversity of Representation
Brian Lester, Daniel Pressel, Amy Hemmeter +2
Most state-of-the-art models in natural language processing (NLP) are neural models built on top of large, pre-trained, contextual language models that generate representations of…
Computationally Efficient NER Taggers with Combined Embeddings and Constrained Decoding
Brian Lester, Daniel Pressel, Amy Hemmeter +1
Current State-of-the-Art models in Named Entity Recognition (NER) are neural models with a Conditional Random Field (CRF) as the final network layer, and pre-trained "contextual em…
Text Extraction and Retrieval from Smartphone Screenshots: Building a Repository for Life in Media
Agnese Chiatti, Mu Jung Cho, Anupriya Gagneja +7
Daily engagement in life experiences is increasingly interwoven with mobile device use. Screen capture at the scale of seconds is being used in behavioral studies and to implement…
Keyword and Keyphrase Extraction Using Centrality Measures on Collocation Networks
Shibamouli Lahiri, Sagnik Ray Choudhury, Cornelia Caragea
Keyword and keyphrase extraction is an important problem in natural language processing, with applications ranging from summarization to semantic search to document clustering. Gra…