6 papers
Humans vs Vision-Language Models: A Unified Measure of Narrative Coherence
Nikolai Ilinykh, Hyewon Jang, Shalom Lappin +2
We study narrative coherence in visually grounded stories by comparing human-written narratives with those generated by vision-language models (VLMs) on the Visual Writing Prompts…
Predicting Sentence Acceptability Judgments in Multimodal Contexts
Hyewon Jang, Nikolai Ilinykh, Sharid Loáiciga +2
Previous work has examined the capacity of deep neural networks (DNNs), particularly transformers, to predict human sentence acceptability judgments, both independently of context,…
Diacritic Restoration for Low-Resource Indigenous Languages: Case Study with Bribri and Cook Islands MÄori
Rolando Coto-Solano, Daisy Li, Manoela Teleginski Ferraz +4
We present experiments on diacritic restoration, a form of text normalization essential for natural language processing (NLP) tasks. Our study focuses on two extremely under-resour…
Coreference as an indicator of context scope in multimodal narrative
Nikolai Ilinykh, Shalom Lappin, Asad Sayeed +1
We demonstrate that large multimodal language models differ substantially from humans in the distribution of coreferential expressions in a visual storytelling task. We introduce a…
Understanding and Analyzing Model Robustness and Knowledge-Transfer in Multilingual Neural Machine Translation using TX-Ray
Vageesh Saxena, Sharid Loáiciga, Nils Rethmeier
Neural networks have demonstrated significant advancements in Neural Machine Translation (NMT) compared to conventional phrase-based approaches. However, Multilingual Neural Machin…
A surprisal oracle for when every layer counts
Xudong Hong, Sharid Loáiciga, Asad Sayeed
Active Curriculum Language Modeling (ACLM; Hong et al., 2023) is a learner directed approach to training a language model. We proposed the original version of this process in our s…