3 citations · 8 across the 14 of their papers we have counts for
14 papers
Models That Know How Evaluations Are Designed Score Safer
Katharina Deckenbach, Haritz Puerto, Jonas Geiping +1
The validity of AI safety evaluations depends on models behaving consistently across controlled and deployment settings. Prior work has identified test-time contextual cues, such a…
From Leaky Thoughts to Private Reasoning: Controlling What LRMs Say to Themselves
Haritz Puerto, Haonan Li, Xudong Han +2
Large reasoning models (LRMs) produce reasoning traces (RTs) that often contain sensitive information. These leaky thoughts are difficult to control and frequently violate explicit…
Leaky Thoughts: Large Reasoning Models Are Not Private Thinkers
Tommaso Green, Martin Gubri, Haritz Puerto +2
We study privacy leakage in the reasoning traces of large reasoning models used as personal agents. Unlike final outputs, reasoning traces are often assumed to be internal and safe…
C-SEO Bench: Does Conversational SEO Work?
Haritz Puerto, Martin Gubri, Tommaso Green +2
Large Language Models (LLMs) are transforming search engines into Conversational Search Engines (CSE). Consequently, Search Engine Optimization (SEO) is being shifted into Conversa…
Scaling Up Membership Inference: When and How Attacks Succeed on Large Language Models
Haritz Puerto, Martin Gubri, Sangdoo Yun +1
Membership inference attacks (MIA) attempt to verify the membership of a given data sample in the training set for a model. MIA has become relevant in recent years, following the r…
Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs
Haritz Puerto, Tilek Chubakov, Xiaodan Zhu +2
Requiring a large language model (LLM) to generate intermediary reasoning steps, known as Chain of Thought (CoT), has been shown to be an effective way of boosting performance. Pre…
Code Prompting Elicits Conditional Reasoning Abilities in Text+Code LLMs
Haritz Puerto, Martin Tutek, Somak Aditya +2
Reasoning is a fundamental component of language understanding. Recent prompting techniques, such as chain of thought, have consistently improved LLMs' performance on various reaso…
UKP-SQuARE: An Interactive Tool for Teaching Question Answering
Haishuo Fang, Haritz Puerto, Iryna Gurevych
The exponential growth of question answering (QA) has made it an indispensable topic in any Natural Language Processing (NLP) course. Additionally, the breadth of QA derived from t…
Surveying (Dis)Parities and Concerns of Compute Hungry NLP Research
Ji-Ung Lee, Haritz Puerto, Betty van Aken +8
Many recent improvements in NLP stem from the development and use of large pre-trained language models (PLMs) with billions of parameters. Large model sizes makes computational cos…
UKP-SQuARE v3: A Platform for Multi-Agent QA Research
Haritz Puerto, Tim Baumgärtner, Rachneet Sachdeva +5
The continuous development of Question Answering (QA) datasets has drawn the research community's attention toward multi-domain models. A popular approach is to use multi-dataset m…
UKP-SQuARE v2: Explainability and Adversarial Attacks for Trustworthy QA
Rachneet Sachdeva, Haritz Puerto, Tim Baumgärtner +6
Question Answering (QA) systems are increasingly deployed in applications where they support real-world decisions. However, state-of-the-art models rely on deep neural networks, wh…
UKP-SQUARE: An Online Platform for Question Answering Research
Tim Baumgärtner, Kexin Wang, Rachneet Sachdeva +10
Recent advances in NLP and information retrieval have given rise to a diverse set of question answering tasks that are of different formats (e.g., extractive, abstractive), require…
MetaQA: Combining Expert Agents for Multi-Skill Question Answering
Haritz Puerto, Gözde Gül Şahin, Iryna Gurevych
The recent explosion of question answering (QA) datasets and models has increased the interest in the generalization of models across multiple domains and formats by either trainin…
Let Me Know What to Ask: Interrogative-Word-Aware Question Generation
Junmo Kang, Haritz Puerto San Roman, Sung-Hyon Myaeng
Question Generation (QG) is a Natural Language Processing (NLP) task that aids advances in Question Answering (QA) and conversational assistants. Existing models focus on generatin…