163 citations · 539 across the 205 of their papers we have counts for
57 papers · 1 filter
Don't Throw Away Your Beams: Improving Consistency-based Uncertainties in LLMs via Beam Search
Ekaterina Fadeeva, Maiya Goloburda, Aleksandr Rubashevskii +5
Consistency-based methods have emerged as an effective approach to uncertainty quantification (UQ) in large language models. These methods typically rely on several generations obt…
EngTrace: A Symbolic Benchmark for Verifiable Process Evaluation of Engineering Reasoning
Ayesha Gull, Muhammad Usman Safder, Rania Elbadry +4
Most engineering problems involve complex multi-step derivations through checkable intermediate quantities. Grading only the final answer misjudges LLMs, crediting correct answers…
PTPP-Aware Adaptation Scaling Laws: Predicting Domain-Adaptation Performance at Unseen Pre-Training Budgets
Etienne Goffinet, Shane Bergsma, Avraham Sheinin +4
Continual pre-training (CPT) for domain adaptation must balance target-domain gains with stability on the base domain. Existing CPT scaling laws typically assume a fixed pre-traini…
HALF: Harm-Aware LLM Fairness Evaluation Aligned with Deployment
Ali Mekky, Omar El Herraoui, Preslav Nakov +1
Large language models (LLMs) are increasingly deployed across high-impact domains, from clinical decision support and legal analysis to hiring and education, making fairness and bi…
BLAZER: Bootstrapping LLM-based Manipulation Agents with Zero-Shot Data Generation
Rocktim Jyoti Das, Harsh Singh, Diana Turmakhan +5
Scaling data and models has played a pivotal role in the remarkable progress of computer vision and language. Inspired by these domains, recent efforts in robotics have similarly f…
DialectalArabicMMLU: Benchmarking Dialectal Capabilities in Arabic and Multilingual Language Models
Malik H. Altakrori, Nizar Habash, Abed Alhakim Freihat +7
We present DialectalArabicMMLU, a new benchmark for evaluating the performance of large language models (LLMs) across Arabic dialects. While recently developed Arabic and multiling…
COMMUNITYNOTES: A Dataset for Exploring the Helpfulness of Fact-Checking Explanations
Rui Xing, Preslav Nakov, Timothy Baldwin +1
Fact-checking on major platforms, such as X, Meta, and TikTok, is shifting from expert-driven verification to a community-based setup, where users contribute explanatory notes to c…
When Personalization Tricks Detectors: The Feature-Inversion Trap in Machine-Generated Text Detection
Lang Gao, Xuhui Li, Chenxi Wang +7
Large language models (LLMs) have grown more powerful in language generation, producing fluent text and even imitating personal style. Yet, this ability also heightens the risk of…
Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation
Jiaying Wu, Zihang Fu, Haonan Wang +4
Community Notes, the crowd-sourced misinformation governance system on X (formerly Twitter), allows users to flag misleading posts, attach contextual notes, and rate the notes' hel…
DUAL-Bench: Measuring Over-Refusal and Robustness in Vision-Language Models
Kaixuan Ren, Preslav Nakov, Usman Naseem
As vision-language models (VLMs) become increasingly capable, maintaining a balance between safety and usefulness remains a central challenge. Safety mechanisms, while essential, c…
Cross-Cultural Transfer of Commonsense Reasoning in LLMs: Evidence from the Arab World
Saeed Almheiri, Rania Hossam, Mena Attia +4
Large language models (LLMs) often reflect Western-centric biases, limiting their effectiveness in diverse cultural contexts. Although some work has explored cultural alignment, th…
Toxicity Red-Teaming: Benchmarking LLM Safety in Singapore's Low-Resource Languages
Yujia Hu, Ming Shan Hee, Preslav Nakov +1
The advancement of Large Language Models (LLMs) has transformed natural language processing; however, their safety mechanisms remain under-explored in low-resource, multilingual se…
Explicit and Implicit Data Augmentation for Social Event Detection
Congbo Ma, Yuxia Wang, Jia Wu +6
Social event detection involves identifying and categorizing important events from social media, which relies on labeled data, but annotation is costly and labor-intensive. To addr…
MuSLR: Multimodal Symbolic Logical Reasoning
Jundong Xu, Hao Fei, Yuhui Zhang +8
Multimodal symbolic logical reasoning, which aims to deduce new facts from multimodal input via formal logic, is critical in high-stakes applications such as autonomous driving and…
A Fano-Style Accuracy Upper Bound for LLM Single-Pass Reasoning in Multi-Hop QA
Kaiyang Wan, Lang Gao, Honglin Mu +3
Multi-Hop Question Answering (MHQA) requires integrating dispersed, interdependent evidence through sequential reasoning under noise. This task is challenging for LLMs as they have…
: A Resource Suite for AI-Generated Code Detection
Daniil Orel, Indraneil Paul, Iryna Gurevych +1
In this work, we compile $\textbf{$\texttt{DroidCollection}$}$, the most extensive open data suite for training and evaluating machine-generated code detectors, comprising over a m…
ConspirED: A Dataset for Cognitive Traits of Conspiracy Theories and Large Language Model Safety
Luke Bates, Max Glockner, Preslav Nakov +1
Conspiracy theories erode public trust in science and institutions while resisting debunking by evolving and absorbing counter-evidence. As AI-generated misinformation becomes incr…
Beyond Memorization: Extending Reasoning Depth with Recurrence, Memory and Test-Time Compute Scaling
Ivan Rodkin, Daniil Orel, Konstantin Smirnov +9
Reasoning is a core capability of large language models, yet how multi-step reasoning is learned and executed remains unclear. We study this question in a controlled cellular-autom…
MuDRiC: Multi-Dialect Reasoning for Arabic Commonsense Validation
Kareem Elozeiri, Mervat Abassy, Preslav Nakov +1
Commonsense validation evaluates whether a sentence aligns with everyday human understanding, a critical capability for developing robust natural language understanding systems. Wh…
Beyond "Not Novel Enough": Enriching Scholarly Critique with LLM-Assisted Feedback
Osama Mohammed Afzal, Preslav Nakov, Tom Hope +1
Novelty assessment is a central yet understudied aspect of peer review, particularly in high volume fields like NLP where reviewer capacity is increasingly strained. We present a s…
BALSAM: A Platform for Benchmarking Arabic Large Language Models
Rawan Al-Matham, Kareem Darwish, Raghad Al-Rasheed +40
The impressive advancement of Large Language Models (LLMs) in English has not been matched across all languages. In particular, LLM performance in Arabic lags behind, due to data s…
FRaN-X: FRaming and Narratives-eXplorer
Artur Muratov, Hana Fatima Shaikh, Vanshikaa Jani +21
We present FRaN-X, a Framing and Narratives Explorer that automatically detects entity mentions and classifies their narrative roles directly from raw text. FRaN-X comprises a two-…
Nile-Chat: Egyptian Language Models for Arabic and Latin Scripts
Guokan Shang, Hadi Abdine, Ahmad Chamma +7
We introduce Nile-Chat-4B, 3x4B-A6B, and 12B, a collection of LLMs for Egyptian dialect, uniquely designed to understand and generate texts written in both Arabic and Latin scripts…
UnsafeChain: Enhancing Reasoning Model Safety via Hard Cases
Raj Vardhan Tomar, Preslav Nakov, Yuxia Wang
As large reasoning models (LRMs) grow more capable, chain-of-thought (CoT) reasoning introduces new safety challenges. Existing SFT-based safety alignment studies dominantly focuse…