6 papers
Auditing Black-Box LLM APIs with a Rank-Based Uniformity Test
Xiaoyuan Zhu, Yaowen Ye, Tianyi Qiu +6
As API access becomes a primary interface to large language models (LLMs), users often interact with black-box systems that offer little transparency into the deployed model. To re…
Self-Improvement as Coherence Optimization: A Theoretical Account
Tianyi Qiu, Ahmed Hani Ismail, Zhonghao He +1
Can language models improve their accuracy without external supervision? Methods such as debate, bootstrap, and internal coherence maximization achieve this surprising feat, even m…
Martingale Score: An Unsupervised Metric for Bayesian Rationality in LLM Reasoning
Zhonghao He, Tianyi Qiu, Hirokazu Shirado +1
Recent advances in reasoning techniques have substantially improved the performance of large language models (LLMs), raising expectations for their ability to provide accurate, tru…
Representative Social Choice: From Learning Theory to AI Alignment
Tianyi Qiu
Social choice theory is the study of preference aggregation across a population, used both in mechanism design for human agents and in the democratic alignment of language models.…
Language Models Resist Alignment: Evidence From Data Compression
Jiaming Ji, Kaile Wang, Tianyi Qiu +7
Large language models (LLMs) may exhibit unintended or undesirable behaviors. Recent works have concentrated on aligning LLMs to mitigate harmful outputs. Despite these efforts, so…
The Lock-in Hypothesis: Stagnation by Algorithm
Tianyi Alex Qiu, Zhonghao He, Tejasveer Chugh +1
The training and deployment of large language models (LLMs) create a feedback loop with human users: models learn human beliefs from data, reinforce these beliefs with generated co…