12 papers
Auction-Based Regulation for Artificial Intelligence
Marco Bornstein, Zora Che, Suhas Julapalli +3
In an era of "moving fast and breaking things", regulators have moved slowly to pick up the safety, bias, and legal debris left in the wake of broken Artificial Intelligence (AI) d…
An Independent Safety Evaluation of Kimi K2.5
Zheng-Xin Yong, Parv Mahajan, Andy Wang +12
Kimi K2.5 is an open-weight LLM that rivals closed models across coding, multimodal, and agentic benchmarks, but was released without an accompanying safety evaluation. In this wor…
Best Practices for Biorisk Evaluations on Open-Weight Bio-Foundation Models
Boyi Wei, Zora Che, Nathaniel Li +10
Open-weight bio-foundation models present a dual-use dilemma. While holding great promise for accelerating scientific research and drug development, they could also enable bad acto…
TutorBench: A Benchmark To Assess Tutoring Capabilities Of Large Language Models
Rakshith S Srinivasa, Zora Che, Chen Bo Calvin Zhang +11
As students increasingly adopt large language models (LLMs) as learning aids, it is crucial to build models that are adept at handling the nuances of tutoring: they need to identif…
Effort-aware Fairness: Incorporating a Philosophy-informed, Human-centered Notion of Effort into Algorithmic Fairness Metrics
Tin Trung Nguyen, Jiannan Xu, Zora Che +6
Although popularized AI fairness metrics, e.g., demographic parity, have uncovered bias in AI-assisted decision-making outcomes, they do not consider how much effort one has spent…
Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities
Zora Che, Stephen Casper, Robert Kirk +12
Evaluations of large language model (LLM) risks and capabilities are increasingly being incorporated into AI risk management and governance frameworks. Currently, most risk evaluat…