4 papers
Mapping Overlaps in Benchmarks through Perplexity in the Wild
Siyang Wu, Honglin Bao, Sida Li +2
We introduce benchmark signatures to characterize the capacity demands of LLM benchmarks and their overlaps. Signatures are sets of salient tokens from in-the-wild corpora whose mo…
LLM Probability Concentration: How Alignment Shrinks the Generative Horizon
Chenghao Yang, Sida Li, Ari Holtzman
Despite their impressive capabilities, aligned large language models (LLMs) often generate outputs that lack diversity. What drives this consistency in the generation? We investiga…
Dynamic Weight Grafting: Localizing Finetuned Factual Knowledge in Transformers
Todd Nief, David Reber, Sean Richardson +1
When an LLM learns a new fact during finetuning (e.g., new movie releases, newly elected pope, etc.), where does this information go? Are entities enriched with relation informatio…
LL3M: Large Language 3D Modelers
Sining Lu, Guan Chen, Nam Anh Dinh +3
We present LL3M, a multi-agent system that leverages pretrained large language models (LLMs) to generate 3D assets by writing interpretable Python code in Blender. We break away fr…