The Geometry of Culture: Analyzing Meaning through Word Embeddings

Abstract

We demonstrate the utility of a new methodological tool, neural-network word embedding models, for large-scale text analysis, revealing how these models produce richer insights into cultural associations and categories than possible with prior methods. Word embeddings represent semantic relations between words as geometric relationships between vectors in a high-dimensional space, operationalizing a relational model of meaning consistent with contemporary theories of identity and culture. We show that dimensions induced by word differences (e.g. man - woman, rich - poor, black - white, liberal - conservative) in these vector spaces closely correspond to dimensions of cultural meaning, and the projection of words onto these dimensions reflects widely shared cultural connotations when compared to surveyed responses and labeled historical data. We pilot a method for testing the stability of these associations, then demonstrate applications of word embeddings for macro-cultural investigation with a longitudinal analysis of the coevolution of gender and class associations in the United States over the 20th century and a comparative analysis of historic distinctions between markers of gender and class in the U.S. and Britain. We argue that the success of these high-dimensional models motivates a move towards "high-dimensional theorizing" of meanings, identities and cultural processes.

References in corpus (8)

• Efficient Estimation of Word Representations in Vector Space
• Maps of random walks on complex networks reveal community structure
• Semantics derived automatically from language corpora contain human-like biases
• Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes
• Characterizing the Google Books corpus: Strong limits to inferences of socio-cultural and linguistic evolution
• Neural Embeddings of Graphs in Hyperbolic Space
• Not All Neural Embeddings are Born Equal
• Parallelizing Word2Vec in Shared and Distributed Memory

Cited by in corpus (23)

• Cultural Cartography with Word Embeddings
• Using Sequences of Life-events to Predict Human Lives
• Principled approach to the selection of the embedding dimension of networks
• Neutral bots probe political bias on social media
• Wide range screening of algorithmic bias in word embedding models using large sentiment lexicons reveals underreported bias types
• FrameAxis: Characterizing Microframe Bias and Intensity with Word Embedding
• Semantic and Relational Spaces in Science of Science: Deep Learning Models for Article Vectorisation
• Unsupervised embedding of trajectories captures the latent structure of scientific migration
• Relating Word Embedding Gender Biases to Gender Gaps: A Cross-Cultural Analysis
• Identification of Biased Terms in News Articles by Comparison of Outlet-specific Word Embeddings
• Human Languages with Greater Information Density Increase Communication Speed, but Decrease Conversation Breadth
• Quantifying social organization and political polarization in online platforms
• Evaluating Biased Attitude Associations of Language Models in an Intersectional Context
• Being Together in Place as a Catalyst for Scientific Advance
• How emoji and word embedding helps to unveil emotional transitions during online messaging
• A Method for Estimating Individual Socioeconomic Status of Twitter Users
• Concept-Guided Chain-of-Thought Prompting for Pairwise Comparison Scoring of Texts with Large Language Models
• Computational Hermeneutics: Evaluating generative AI as a cultural technology
• Constructing Vec-tionaries to Extract Message Features from Texts: A Case Study of Moral Appeals
• Discovering and Interpreting Biased Concepts in Online Communities
• Labor Space: A Unifying Representation of the Labor Market via Large Language Models
• The presence of occupational structure in online texts based on word embedding NLP models
• Agree to Disagree: Measuring Hidden Dissent in FOMC Meetings