565 citations · 1.2k across the 48 of their papers we have counts for
49 papers
The Future of NLP may not be at NLP Conferences: Scholarly Migration Patterns in Natural Language Processing
David Jurgens
Natural Language Processing (NLP) has traditionally been published in its core disciplinary venues like ACL. However, advances in Large Language Models (LLMs) has led to a blurring…
Real or Robotic? Assessing Whether LLMs Accurately Simulate Qualities of Human Responses in Dialogue
Jonathan Ivey, Shivani Kumar, Jiayu Liu +12
Studying and building datasets for dialogue tasks is both expensive and time-consuming due to the need to recruit, train, and collect data from study participants. In response, muc…
The Role of Network and Identity in the Diffusion of Hashtags
Aparna Ananthasubramaniam, Yufei 'Louise' Zhu, David Jurgens +1
The diffusion of culture online is theorized to be influenced by many interacting social factors (e.g., network and identity). However, most existing computational cascade models c…
A Multilingual Similarity Dataset for News Article Frame
Xi Chen, Mattia Samory, Scott Hale +2
Understanding the writing frame of news articles is vital for addressing social issues, and thus has attracted notable attention in the fields of communication studies. Yet, assess…
The Call for Socially Aware Language Technologies
Diyi Yang, Dirk Hovy, David Jurgens +1
Language technologies have made enormous progress, especially with the introduction of large language models (LLMs). On traditional tasks such as machine translation and sentiment…
Global News Synchrony and Diversity During the Start of the COVID-19 Pandemic
Xi Chen, Scott A. Hale, David Jurgens +3
News coverage profoundly affects how countries and individuals behave in international relations. Yet, we have little empirical evidence of how news coverage varies across countrie…
When it Rains, it Pours: Modeling Media Storms and the News Ecosystem
Benjamin Litterer, David Jurgens, Dallas Card
Most events in the world receive at most brief coverage by the news media. Occasionally, however, an event will trigger a media storm, with voluminous and widespread coverage lasti…
Social Meme-ing: Measuring Linguistic Variation in Memes
Naitian Zhou, David Jurgens, David Bamman
Much work in the space of NLP has used computational methods to explore sociolinguistic variation in text. In this paper, we argue that memes, as multimodal forms of language compr…
When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models
Mingqian Zheng, Jiaxin Pei, Lajanugen Logeswaran +2
Prompting serves as the major way humans interact with Large Language Models (LLM). Commercial AI systems commonly define the role of the LLM in system prompts. For example, ChatGP…
Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMs
Huaman Sun, Jiaxin Pei, Minje Choi +1
Human judgments are inherently subjective and are actively affected by personal traits such as gender and ethnicity. While Large Language Models (LLMs) are widely used to simulate…
You don't need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments
Bangzhao Shu, Lechen Zhang, Minje Choi +5
The versatility of Large Language Models (LLMs) on natural language understanding tasks has made them popular for research in social sciences. To properly understand the properties…
Exploring Linguistic Style Matching in Online Communities: The Role of Social Context and Conversation Dynamics
Aparna Ananthasubramaniam, Hong Chen, Jason Yan +7
Linguistic style matching (LSM) in conversations can be reflective of several aspects of social influence such as power or persuasion. However, how LSM relates to the outcomes of o…
Profile Update: The Effects of Identity Disclosure on Network Connections and Language
Minje Choi, Daniel M. Romero, David Jurgens
Our social identities determine how we interact and engage with the world surrounding us. In online settings, individuals can make these identities explicit by including them in th…
Your spouse needs professional help: Determining the Contextual Appropriateness of Messages through Modeling Social Relationships
David Jurgens, Agrima Seth, Jackson Sargent +2
Understanding interpersonal communication requires, in part, understanding the social context and norms in which a message is said. However, current methods for identifying offensi…
RCT Rejection Sampling for Causal Estimation Evaluation
Katherine A. Keith, Sergey Feldman, David Jurgens +2
Confounding is a significant obstacle to unbiased estimation of causal effects from observational data. For settings with high-dimensional covariates -- such as text data, genomics…
When Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Dataset
Jiaxin Pei, David Jurgens
Annotators are not fungible. Their demographics, life experiences, and backgrounds all contribute to how they label data. However, NLP has only recently considered how annotator id…
Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark
Minje Choi, Jiaxin Pei, Sagar Kumar +2
Large language models (LLMs) have been shown to perform well at a variety of syntactic, discourse, and reasoning tasks. While LLMs are increasingly deployed in many forms including…
Bridging Nations: Quantifying the Role of Multilinguals in Communication on Social Media
Julia Mendelsohn, Sayan Ghosh, David Jurgens +1
Social media enables the rapid spread of many kinds of information, from memes to social movements. However, little is known about how information crosses linguistic boundaries. We…
Analyzing the Engagement of Social Relationships During Life Event Shocks in Social Media
Minje Choi, David Jurgens, Daniel M. Romero
Individuals experiencing unexpected distressing events, shocks, often rely on their social network for support. While prior work has shown how social networks respond to shocks, th…
POTATO: The Portable Text Annotation Tool
Jiaxin Pei, Aparna Ananthasubramaniam, Xingyao Wang +4
We present POTATO, the Portable text annotation tool, a free, fully open-sourced annotation system that 1) supports labeling many types of text and multimodal data; 2) offers easy-…
A Critical Reflection and Forward Perspective on Empathy and Natural Language Processing
Allison Lahnala, Charles Welch, David Jurgens +1
We review the state of research on empathy in natural language processing and identify the following issues: (1) empathy definitions are absent or abstract, which (2) leads to low…
Modeling Information Change in Science Communication with Semantically Matched Paraphrases
Dustin Wright, Jiaxin Pei, David Jurgens +1
Whether the media faithfully communicate scientific information has long been a core issue to the science community. Automatically identifying paraphrased scientific findings could…
SemEval 2023 Task 9: Multilingual Tweet Intimacy Analysis
Jiaxin Pei, Vítor Silva, Maarten Bos +4
We propose MINT, a new Multilingual INTimacy analysis dataset covering 13,372 tweets in 10 languages including English, French, Spanish, Italian, Portuguese, Korean, Dutch, Chinese…
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabil…