5 papers
Politically Speaking: LLMs on Changing International Affairs
Xuenan Cao, Wai Kei Chung, Ye Zhao +1
Ask your chatbot to impersonate an expert from Russia and an expert from US and query it on Chinese politics. How might the outputs differ? Or, to prepare ourselves for the worse,…
Advocate for Complete Benchmarks for Formal Reasoning with Formal/Informal Statements and Formal/Informal Proofs
Roozbeh Yousefzadeh, Xuenan Cao
This position paper provides a critical but constructive discussion of current practices in benchmarking and evaluative practices in the field of formal reasoning and automated the…
A Lean Dataset for International Math Olympiad: Small Steps towards Writing Math Proofs for Hard Problems
Roozbeh Yousefzadeh, Xuenan Cao, Azim Ospanov
Using AI to write formal proofs for mathematical problems is a challenging task that has seen some advancements in recent years. Automated systems such as Lean can verify the corre…
How Large Language Models (LLMs) Extrapolate: From Guided Missiles to Guided Prompts
Xuenan Cao
This paper argues that we should perceive LLMs as machines of extrapolation. Extrapolation is a statistical function for predicting the next value in a series. Extrapolation contri…
Towards a Scalable Reference-Free Evaluation of Generative Models
Azim Ospanov, Jingwei Zhang, Mohammad Jalali +3
While standard evaluation scores for generative models are mostly reference-based, a reference-dependent assessment of generative models could be generally difficult due to the una…