Generalized Word Shift Graphs: A Method for Visualizing and Explaining Pairwise Comparisons Between Texts

Abstract

A common task in computational text analyses is to quantify how two corpora differ according to a measurement like word frequency, sentiment, or information content. However, collapsing the texts' rich stories into a single number is often conceptually perilous, and it is difficult to confidently interpret interesting or unexpected textual patterns without looming concerns about data artifacts or measurement validity. To better capture fine-grained differences between texts, we introduce generalized word shift graphs, visualizations which yield a meaningful and interpretable summary of how individual words contribute to the variation between two texts for any measure that can be formulated as a weighted average. We show that this framework naturally encompasses many of the most commonly used approaches for comparing texts, including relative frequencies, dictionary scores, and entropy-based measures like the Kullback-Leibler and Jensen-Shannon divergences. Through several case studies, we demonstrate how generalized word shift graphs can be flexibly applied across domains for diagnostic investigation, hypothesis generation, and substantive interpretation. By providing a detailed lens into textual shifts between corpora, generalized word shift graphs help computational social scientists, digital humanists, and other text analysis practitioners fashion more robust scientific narratives.

arXiv comment20 pages, 7 figures, 2 tables

References in corpus (15)

• Temporal patterns of happiness and information in a global social network: Hedonometrics and Twitter
• The Geography of Happiness: Connecting Twitter sentiment and expression, demographics, and objective characteristics of place
• Bots increase exposure to negative and inflammatory content in online social systems
• Characterizing the Google Books corpus: Strong limits to inferences of socio-cultural and linguistic evolution
• The emotional arcs of stories are dominated by six basic shapes
• Measuring the happiness of large-scale written expression: Songs, Blogs, and Presidents
• Weather impacts expressed sentiment
• Divergent discourse between protests and counter-protests: #BlackLivesMatter and #AllLivesMatter
• Statistical Laws Governing Fluctuations in Word Use from Word Birth to Word Death
• Individuals, Institutions, and Innovation in the Debates of the French Revolution
• Visitors to urban greenspace have higher sentiment and lower negativity on Twitter
• Small cities face greater impact from automation
• Generalized Entropies and the Similarity of Texts
• Allotaxonometry and rank-turbulence divergence: A universal instrument for comparing complex systems
• Gauging the happiness benefit of US urban parks through Twitter

Cited by in corpus (11)

• FrameAxis: Characterizing Microframe Bias and Intensity with Word Embedding
• Hurricanes Increase Climate Change Conversations on Twitter
• The Impact of Data Persistence Bias on Social Media Studies
• Augmenting semantic lexicons using word embeddings and transfer learning
• LibertyMFD: A Lexicon to Assess the Moral Foundation of Liberty
• Quantifying language changes surrounding mental health on Twitter
• Wikibio: a Semantic Resource for the Intersectional Analysis of Biographical Events
• When the Echo Chamber Shatters: Examining the Use of Community-Specific Language Post-Subreddit Ban
• From Flowers to Fascism? The Cottagecore to Tradwife Pipeline on Tumblr
• MNet-Sim: A Multi-layered Semantic Similarity Network to Evaluate Sentence Similarity
• Ousiometrics: The essence of meaning aligns with a power-danger-structure framework instead of valence-arousal-dominance