23 citations · 33 across the 4 of their papers we have counts for
8 papers · 1 filter
Measuring Scalar Constructs in Social Science with LLMs
Hauke Licht, Rupak Sarkar, Patrick Y. Wu +4
Many constructs that characterize language, like its complexity or emotionality, have a naturally continuous semantic structure; a public speech is not just "simple" or "complex,"…
Improving the TENOR of Labeling: Re-evaluating Topic Models for Content Analysis
Zongxia Li, Andrew Mao, Daniel Stephens +5
Topic models are a popular tool for understanding text collections, but their evaluation has been a point of contention. Automated evaluation metrics such as coherence are often us…
Natural Language Decompositions of Implicit Content Enable Better Text Representations
Alexander Hoyle, Rupak Sarkar, Pranav Goel +1
When people interpret text, they rely on inferences that go beyond the observed language itself. Inspired by this observation, we introduce a method for the analysis of text that t…
Are Neural Topic Models Broken?
Alexander Hoyle, Pranav Goel, Rupak Sarkar +1
Recently, the relationship between automated and human evaluation of topic models has been called into question. Method developers have staked the efficacy of new topic model varia…
Studying word order through iterative shuffling
Nikolay Malkin, Sameera Lanka, Pranav Goel +1
As neural language models approach human performance on NLP benchmark tasks, their advances are widely seen as evidence of an increasingly complex understanding of syntax. This vie…
Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence
Alexander Hoyle, Pranav Goel, Denis Peskov +3
Topic model evaluation, like evaluation of other unsupervised methods, can be contentious. However, the field has coalesced around automated estimates of topic coherence, which rel…