5 citations · 40 across the 105 of their papers we have counts for
36 papers · 1 filter
Libra-Leaderboard: Towards Responsible AI through a Balanced Leaderboard of Safety and Capability
Haonan Li, Xudong Han, Zenan Zhai +32
To address this gap, we introduce Libra-Leaderboard, a comprehensive framework designed to rank LLMs through a balanced evaluation of performance and safety. Combining a dynamic le…
FIRE: Fact-checking with Iterative Retrieval and Verification
Zhuohan Xie, Rui Xing, Yuxia Wang +5
Fact-checking long-form text is challenging, and it is therefore common practice to break it down into multiple atomic claims. The typical approach to fact-checking these atomic cl…
Are Large Language Models Good Classifiers? A Study on Edit Intent Classification in Scientific Document Revisions
Qian Ruan, Ilia Kuznetsov, Iryna Gurevych
Classification is a core NLP task architecture with many potential applications. While large language models (LLMs) have brought substantial advancements in text generation, their…
Transforming Scholarly Landscapes: Influence of Large Language Models on Academic Fields beyond Computer Science
Aniket Pramanick, Yufang Hou, Saif M. Mohammad +1
Large Language Models (LLMs) have ushered in a transformative era in Natural Language Processing (NLP), reshaping research and extending NLP's influence to other fields of study. H…
Efficient Performance Tracking: Leveraging Large Language Models for Automated Construction of Scientific Leaderboards
Furkan Şahinuç, Thy Thy Tran, Yulia Grishina +3
Scientific leaderboards are standardized ranking systems that facilitate evaluating and comparing competitive methods. Typically, a leaderboard is defined by a task, dataset, and e…
The Nature of NLP: Analyzing Contributions in NLP Papers
Aniket Pramanick, Yufang Hou, Saif M. Mohammad +1
Natural Language Processing (NLP) is an established and dynamic field. Despite this, what constitutes NLP research remains debated. In this work, we address the question by quantit…
RIRAG: Regulatory Information Retrieval and Answer Generation
Tuba Gokhan, Kexin Wang, Iryna Gurevych +1
Regulatory documents, issued by governmental regulatory bodies, establish rules, guidelines, and standards that organizations must adhere to for legal compliance. These documents,…
STRICTA: Structured Reasoning in Critical Text Assessment for Peer Review and Beyond
Nils Dycke, Matej Zečević, Ilia Kuznetsov +3
Critical text assessment is at the core of many expert activities, such as fact-checking, peer review, and essay grading. Yet, existing work treats critical text assessment as a bl…
"Image, Tell me your story!" Predicting the original meta-context of visual misinformation
Jonathan Tonglet, Marie-Francine Moens, Iryna Gurevych
To assist human fact-checkers, researchers have developed automated approaches for visual misinformation detection. These methods assign veracity scores by identifying inconsistenc…
DOCE: Finding the Sweet Spot for Execution-Based Code Generation
Haau-Sing Li, Patrick Fernandes, Iryna Gurevych +1
Recently, a diverse set of decoding and reranking procedures have been shown effective for LLM-based code generation. However, a comprehensive framework that links and experimental…
Grounding Fallacies Misrepresenting Scientific Publications in Evidence
Max Glockner, Yufang Hou, Preslav Nakov +1
Health-related misinformation claims often falsely cite a credible biomedical publication as evidence. These publications only superficially seem to support the false claim, when l…
LLM-DetectAIve: a Tool for Fine-Grained Machine-Generated Text Detection
Mervat Abassy, Kareem Elozeiri, Alexander Aziz +21
The ease of access to large language models (LLMs) has enabled a widespread of machine-generated texts, and now it is often hard to tell whether a piece of text was human-written o…
A Course Shared Task on Evaluating LLM Output for Clinical Questions
Yufang Hou, Thy Thy Tran, Doan Nam Long Vu +4
This paper presents a shared task that we organized at the Foundations of Language Technology (FoLT) course in 2023/2024 at the Technical University of Darmstadt, which focuses on…
: Towards Measuring Class-wise Hardness through Modelling Class Semantics
Fengyu Cai, Xinran Zhao, Hongming Zhang +2
Recent advances in measuring hardness-wise properties of data guide language models in sample selection within low-resource scenarios. However, class-specific properties are overlo…
HDT: Hierarchical Document Transformer
Haoyu He, Markus Flicke, Jan Buchmann +2
In this paper, we propose the Hierarchical Document Transformer (HDT), a novel sparse Transformer architecture tailored for structured hierarchical documents. Such documents are ex…
Systematic Task Exploration with LLMs: A Study in Citation Text Generation
Furkan Şahinuç, Ilia Kuznetsov, Yufang Hou +1
Large language models (LLMs) bring unprecedented flexibility in defining and executing complex, creative natural language generation (NLG) tasks. Yet, this flexibility brings new c…
Localizing and Mitigating Errors in Long-form Question Answering
Rachneet Sachdeva, Yixiao Song, Mohit Iyyer +1
Long-form question answering (LFQA) aims to provide thorough and in-depth answers to complex questions, enhancing comprehension. However, such detailed responses are prone to hallu…
Preemptive Detection and Correction of Misaligned Actions in LLM Agents
Haishuo Fang, Xiaodan Zhu, Iryna Gurevych
Deploying LLM-based agents in real-life applications often faces a critical challenge: the misalignment between agents' behavior and user intent. Such misalignment may lead agents…
Robust Utility-Preserving Text Anonymization Based on Large Language Models
Tianyu Yang, Xiaodan Zhu, Iryna Gurevych
Anonymizing text that contains sensitive information is crucial for a wide range of applications. Existing techniques face the emerging challenges of the re-identification ability…
: Enhancing Retriever Generalization for Scientific Domain through Complementary Granularity
Fengyu Cai, Xinran Zhao, Tong Chen +4
Recent studies show the growing significance of document retrieval in the generation of LLMs, i.e., RAG, within the scientific domain by bridging their knowledge gap. However, dens…
Attribute or Abstain: Large Language Models as Long Document Assistants
Jan Buchmann, Xiao Liu, Iryna Gurevych
LLMs can help humans working with long documents, but are known to hallucinate. Attribution can increase trust in LLM responses: The LLM provides evidence that supports its respons…
LLM Roleplay: Simulating Human-Chatbot Interaction
Hovhannes Tamoyan, Hendrik Schuff, Iryna Gurevych
The development of chatbots requires collecting a large number of human-chatbot dialogues to reflect the breadth of users' sociodemographic backgrounds and conversational goals. Ho…
Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs
Haritz Puerto, Tilek Chubakov, Xiaodan Zhu +2
Requiring a large language model (LLM) to generate intermediary reasoning steps, known as Chain of Thought (CoT), has been shown to be an effective way of boosting performance. Pre…
M2QA: Multi-domain Multilingual Question Answering
Leon Engländer, Hannah Sterz, Clifton Poth +3
Generalization and robustness to input variation are core desiderata of machine learning research. Language varies along several axes, most importantly, language instance (e.g. Fre…