5 papers
Reward Modeling for Scientific Writing Evaluation
Furkan Şahinuç, Subhabrata Dutta, Iryna Gurevych
Scientific writing is an expert-domain task that demands deep domain knowledge, task-specific requirements and reasoning capabilities that leverage the domain knowledge to satisfy…
Expert Preference-based Evaluation of Automated Related Work Generation
Furkan Şahinuç, Subhabrata Dutta, Iryna Gurevych
Expert domain writing, such as scientific writing, typically demands extensive domain knowledge. Although large language models (LLMs) show promising potential in this task, evalua…
Efficient Performance Tracking: Leveraging Large Language Models for Automated Construction of Scientific Leaderboards
Furkan Şahinuç, Thy Thy Tran, Yulia Grishina +3
Scientific leaderboards are standardized ranking systems that facilitate evaluating and comparing competitive methods. Typically, a leaderboard is defined by a task, dataset, and e…
Systematic Task Exploration with LLMs: A Study in Citation Text Generation
Furkan Şahinuç, Ilia Kuznetsov, Yufang Hou +1
Large language models (LLMs) bring unprecedented flexibility in defining and executing complex, creative natural language generation (NLG) tasks. Yet, this flexibility brings new c…
Imparting Interpretability to Word Embeddings while Preserving Semantic Structure
Lutfi Kerem Senel, Ihsan Utlu, Furkan Şahinuç +2
As an ubiquitous method in natural language processing, word embeddings are extensively employed to map semantic properties of words into a dense vector representation. They captur…