12 papers
Is Convergence Inevitable? Tracing Output Homogeneity Back to Base Models
Alexandrine Fortier, Hazel Chen, Peter West
The lack of diversity in LM content is widely attributed to the alignment process, but how and where exactly in the pipeline this collapse begins is unknown. We argue that output h…
UnpredictaBench: A Benchmark for Evaluating Distributional Randomness in LLMs
Amirhossein Abaskohi, Amirhossein Dabiriaghdam, Liang Luo +4
We introduce UnpredictaBench, an evaluation that tests the ability of large language models (LLMs) to capture true underlying distributions. As LLMs are increasingly used as substi…
SeKV: Resolution-Adaptive KV Cache with Hierarchical Semantic Memory for Long-Context LLM Inference
Amirhossein Abaskohi, Giuseppe Carenini, Peter West +1
Large language models increasingly operate over long contexts, where the KV cache becomes a dominant memory bottleneck: its size grows linearly with sequence length and must be ret…
Towards Physical Intuitions for Alignment Dynamics: A Case Study With Randomness Crystallization
Kunal Samanta, Ari Holtzman, Peter West
The alignment of language models is typically studied through the lens of capability benchmarks, but the dynamics of how models change during post-training remain poorly understood…
Resolution Thresholds in VLM Detection of Harmful ASCII Art Across Construction Modes and Languages
Yikai Hua, Peter West
Large Vision-Language Models (VLMs) are increasingly deployed as content moderation tools, yet they remain vulnerable to jailbreak attacks in which harmful text is visually encoded…
MCompassRAG: Topic Metadata as a Semantic Compass for Paragraph-Level Retrieval
Amirhossein Abaskohi, Raymond Li, Gaetano Cimino +3
Retrieval-augmented generation (RAG) systems depend critically on how documents are chunked and searched. Fine-grained chunks can improve retrieval precision but expand the search…