11 papers
The Plot Thins: Uniformity and Linearity in Literary Summaries
Rebecca M. M. Hicke, Sil Hamilton, David Mimno +1
Works of literature are complicated; they balance plot, suspense, surprise, and artistic expression. Summaries of literature prioritize plot, and therefore may deviate from their s…
Elias in the Lighthouse, Again? Diagnosing Low Diversity in LLM Stories
Sil Hamilton, David Mimno
LLM-generated stories are a popular use case, but they show very low variability. We sample 20,000 total stories from four current models using five prompts. We find that 11 words…
Do Chinese models speak Chinese languages?
Andrea W Wen-Yi, Unso Eun Seo Jo, David Mimno
The release of top-performing open-weight LLMs has cemented China's role as a leading force in AI development. Do these models support languages spoken in China? Or do they support…
Attention Flows: Tracing LLM Conceptual Engagement via Story Summaries
Rebecca M. M. Hicke, Sil Hamilton, David Mimno +1
Although LLM context lengths have grown, there is evidence that their ability to integrate information across long-form texts has not kept pace. We evaluate one such understanding…
Are You There God? Lightweight Narrative Annotation of Christian Fiction with LMs
Rebecca M. M. Hicke, Brian W. Haggard, Mia Ferrante +2
In addition to its more widely studied cultural movements, American Evangelicalism has a well-developed but less externally visible literary side. Christian Fiction, however, has b…
How Chinese are Chinese Language Models? The Puzzling Lack of Language Policy in China's LLMs
Andrea W Wen-Yi, Unso Eun Seo Jo, Lu Jia Lin +1
Contemporary language models are increasingly multilingual, but Chinese LLM developers must navigate complex political and business considerations of language diversity. Language p…