17 citations · 31 across the 20 of their papers we have counts for
23 papers
Is Peer Review Really in Decline? Analyzing Review Quality across Venues and Time
Ilia Kuznetsov, Rohan Nayak, Alla Rozovskaya +1
Peer review is at the heart of modern science. As submission numbers rise and research communities grow, the decline in review quality is a popular narrative and a common concern.…
Exposía: Teaching and Assessment of Academic Writing Skills for Research Project Proposals and Peer Feedback
Dennis Zyska, Alla Rozovskaya, Ilia Kuznetsov +1
We present Exposía, the first public dataset that connects writing and feedback in higher education, enabling research on educationally grounded computational approaches to teachin…
ABCD-LINK: Annotation Bootstrapping for Cross-Document Fine-Grained Links
Serwar Basch, Ilia Kuznetsov, Tom Hope +1
Understanding fine-grained links between documents is crucial for many applications, yet progress is limited by the lack of efficient methods for data curation. To address this lim…
Pull Requests From The Classroom: Co-Developing Curriculum And Code
Dennis Zyska, Ilia Kuznetsov, Florian Müller +1
Educational technologies often misalign with instructors' pedagogical goals, forcing adaptations that compromise teaching efficacy. In this paper, we present a case study on the co…
Identifying Aspects in Peer Reviews
Sheng Lu, Ilia Kuznetsov, Iryna Gurevych
Peer review is central to academic publishing, but the growing volume of submissions is straining the process. This motivates the development of computational approaches to support…
Are Large Language Models Good Classifiers? A Study on Edit Intent Classification in Scientific Document Revisions
Qian Ruan, Ilia Kuznetsov, Iryna Gurevych
Classification is a core NLP task architecture with many potential applications. While large language models (LLMs) have brought substantial advancements in text generation, their…
STRICTA: Structured Reasoning in Critical Text Assessment for Peer Review and Beyond
Nils Dycke, Matej Zečević, Ilia Kuznetsov +3
Critical text assessment is at the core of many expert activities, such as fact-checking, peer review, and essay grading. Yet, existing work treats critical text assessment as a bl…
Systematic Task Exploration with LLMs: A Study in Citation Text Generation
Furkan Şahinuç, Ilia Kuznetsov, Yufang Hou +1
Large language models (LLMs) bring unprecedented flexibility in defining and executing complex, creative natural language generation (NLG) tasks. Yet, this flexibility brings new c…
M2QA: Multi-domain Multilingual Question Answering
Leon Engländer, Hannah Sterz, Clifton Poth +3
Generalization and robustness to input variation are core desiderata of machine learning research. Language varies along several axes, most importantly, language instance (e.g. Fre…
Re3: A Holistic Framework and Dataset for Modeling Collaborative Document Revision
Qian Ruan, Ilia Kuznetsov, Iryna Gurevych
Collaborative review and revision of textual documents is the core of knowledge work and a promising target for empirical analysis and NLP assistance. Yet, a holistic framework tha…
What Can Natural Language Processing Do for Peer Review?
Ilia Kuznetsov, Osama Mohammed Afzal, Koen Dercksen +21
The number of scientific articles produced every year is growing rapidly. Providing quality control over them is crucial for scientists and, ultimately, for the public good. In mod…
Document Structure in Long Document Transformers
Jan Buchmann, Max Eichler, Jan-Micha Bodensohn +2
Long documents often exhibit structure with hierarchically organized elements of different functions, such as section headers and paragraphs. Despite the omnipresence of document s…
CARE: Collaborative AI-Assisted Reading Environment
Dennis Zyska, Nils Dycke, Jan Buchmann +2
Recent years have seen impressive progress in AI-assisted writing, yet the developments in AI-assisted reading are lacking. We propose inline commentary as a natural vehicle for AI…
CiteBench: A benchmark for Scientific Citation Text Generation
Martin Funkquist, Ilia Kuznetsov, Yufang Hou +1
Science progresses by building upon the prior body of knowledge documented in scientific publications. The acceleration of research makes it hard to stay up-to-date with the recent…
NLPeer: A Unified Resource for the Computational Study of Peer Review
Nils Dycke, Ilia Kuznetsov, Iryna Gurevych
Peer review constitutes a core component of scholarly publishing; yet it demands substantial expertise and training, and is susceptible to errors and biases. Various applications o…
An Inclusive Notion of Text
Ilia Kuznetsov, Iryna Gurevych
Natural language processing (NLP) researchers develop models of grammar, meaning and communication based on written text. Due to task and data differences, what is considered text…
Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Review
Ilia Kuznetsov, Jan Buchmann, Max Eichler +1
Peer review is a key component of the publishing process in most fields of science. The increasing submission rates put a strain on reviewing quality and efficiency, motivating the…
Yes-Yes-Yes: Proactive Data Collection for ACL Rolling Review and Beyond
Nils Dycke, Ilia Kuznetsov, Iryna Gurevych
The shift towards publicly available text sources has enabled language processing at unprecedented scale, yet leaves under-serviced the domains where public and openly licensed dat…
Assisting Decision Making in Scholarly Peer Review: A Preference Learning Perspective
Nils Dycke, Edwin Simpson, Ilia Kuznetsov +1
Peer review is the primary means of quality control in academia; as an outcome of a peer review process, program and area chairs make acceptance decisions for each paper based on t…
A Matter of Framing: The Impact of Linguistic Formalism on Probing Results
Ilia Kuznetsov, Iryna Gurevych
Deep pre-trained contextualized encoders like BERT (Delvin et al., 2019) demonstrate remarkable performance on a range of downstream tasks. A recent line of research in probing inv…
Does My Rebuttal Matter? Insights from a Major NLP Conference
Yang Gao, Steffen Eger, Ilia Kuznetsov +2
Peer review is a core element of the scientific process, particularly in conference-centered fields such as ML and NLP. However, only few studies have evaluated its properties empi…
LINSPECTOR: Multilingual Probing Tasks for Word Representations
Gözde Gül Şahin, Clara Vania, Ilia Kuznetsov +1
Despite an ever growing number of word representation models introduced for a large number of languages, there is a lack of a standardized technique to provide insights into what i…
EELECTION at SemEval-2017 Task 10: Ensemble of nEural Learners for kEyphrase ClassificaTION
Steffen Eger, Erik-Lân Do Dinh, Ilia Kuznetsov +2
This paper describes our approach to the SemEval 2017 Task 10: "Extracting Keyphrases and Relations from Scientific Publications", specifically to Subtask (B): "Classification of i…