7 papers
Multimodal QUD: Inquisitive Questions from Scientific Figures
Yating Wu, William Rudman, Venkata S Govindarajan +2
Discourse comprehension in complex documents often involves continuously posing and resolving Questions Under Discussion (QUDs). While QUD frameworks have so far focused on text, s…
Dark & Stormy: Modeling Humor in Sentences from the Bulwer-Lytton Fiction Contest
Venkata S Govindarajan, Laura Biester
Textual humor is enormously diverse and computational studies need to account for this range, including intentionally bad humor. In this paper, we curate and analyze a novel corpus…
Measuring Lexical Diversity of Synthetic Data Generated through Fine-Grained Persona Prompting
Gauri Kambhatla, Chantal Shaib, Venkata Govindarajan
Fine-grained personas have recently been used for generating 'diverse' synthetic data for pre-training and supervised fine-tuning of Large Language Models (LLMs). In this work, we…
Do they mean 'us'? Interpreting Referring Expressions in Intergroup Bias
Venkata S Govindarajan, Matianyu Zang, Kyle Mahowald +2
The variations between in-group and out-group speech (intergroup bias) are subtle and could underlie many social phenomena like stereotype perpetuation and implicit bias. In this p…
Counterfactually Probing Language Identity in Multilingual Models
Anirudh Srinivasan, Venkata S Govindarajan, Kyle Mahowald
Techniques in causal analysis of language models illuminate how linguistic information is organized in LLMs. We use one such technique, AlterRep, a method of counterfactual probing…
Lil-Bevo: Explorations of Strategies for Training Language Models in More Humanlike Ways
Venkata S Govindarajan, Juan Diego Rodriguez, Kaj Bostrom +1
We present Lil-Bevo, our submission to the BabyLM Challenge. We pretrained our masked language models with three ingredients: an initial pretraining with music data, training on sh…