3 citations · 5 across the 7 of their papers we have counts for
8 papers
How Important is `Perfect' English for Machine Translation Prompts?
Patrícia Schmidtová, Niyati Bafna, Seth Aycock +4
Large language models (LLMs) have achieved top results in recent machine translation evaluations, but they are also known to be sensitive to errors and perturbations in their promp…
LLMs as Span Annotators: A Comparative Study of LLMs and Humans
Zdeněk Kasner, Vilém Zouhar, Patrícia Schmidtová +7
Span annotation - annotating specific text features at the span level - can be used to evaluate texts where single-score metrics fail to provide actionable feedback. Until recently…
Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practices
Patrícia Schmidtová, Saad Mahamood, Simone Balloccu +6
Automatic metrics are extensively used to evaluate natural language processing systems. However, there has been increasing focus on how they are used and reported by practitioners…
factgenie: A Framework for Span-based Evaluation of Generated Texts
Zdeněk Kasner, Ondřej Plátek, Patrícia Schmidtová +2
We present factgenie: a framework for annotating and visualizing word spans in textual model outputs. Annotations can capture various span-based phenomena such as semantic inaccura…
Three Ways of Using Large Language Models to Evaluate Chat
Ondřej Plátek, Vojtěch Hudeček, Patricia Schmidtová +2
This paper describes the systems submitted by team6 for ChatEval, the DSTC 11 Track 4 competition. We present three different approaches to predicting turn-level qualities of chatb…
DialogueScript: Using Dialogue Agents to Produce a Script
Patrícia Schmidtová, Dávid Javorský, Christián Mikláš +3
We present a novel approach to generating scripts by using agents with different personality types. To manage character interaction in the script, we employ simulated dramatic netw…
THEaiTRE 1.0: Interactive generation of theatre play scripts
Rudolf Rosa, Tomáš Musil, Ondřej Dušek +13
We present the first version of a system for interactive generation of theatre play scripts. The system is based on a vanilla GPT-2 model with several adjustments, targeting specif…
THEaiTRE: Artificial Intelligence to Write a Theatre Play
Rudolf Rosa, Ondřej Dušek, Tom Kocmi +10
We present THEaiTRE, a starting project aimed at automatic generation of theatre play scripts. This paper reviews related work and drafts an approach we intend to follow. We plan t…