4 papers
Death of the Novel(ty): Beyond n-Gram Novelty as a Metric for Textual Creativity
Arkadiy Saakyan, Najoung Kim, Smaranda Muresan +1
N-gram novelty is widely used to evaluate language models' ability to generate text outside of their training data. More recently, it has also been adopted as a metric for measurin…
VideoNorms: Benchmarking Cultural Awareness of Video Language Models
Nikhil Reddy Varimalla, Yunfei Xu, Arkadiy Saakyan +2
As Video Large Language Models (VideoLLMs) are deployed globally, it is important to assess their ability to reason across cultural contexts. To advance cultural norm awareness eva…
Understanding Figurative Meaning through Explainable Visual Entailment
Arkadiy Saakyan, Shreyas Kulkarni, Tuhin Chakrabarty +1
Large Vision-Language Models (VLMs) have demonstrated strong capabilities in tasks requiring a fine-grained understanding of literal meaning in images and text, such as visual ques…
ICLEF: In-Context Learning with Expert Feedback for Explainable Style Transfer
Arkadiy Saakyan, Smaranda Muresan
While state-of-the-art large language models (LLMs) can excel at adapting text from one style to another, current work does not address the explainability of style transfer models.…