1 citations · 1 across the 11 of their papers we have counts for
7 papers · 1 filter
Visualizing token importance for black-box language models
Paulius Rauba, Qiyao Wei, Mihaela van der Schaar
We consider the problem of auditing black-box large language models (LLMs) to ensure they behave reliably when deployed in production settings, particularly in high-stakes domains…
Reasoning Under Pressure: How do Training Incentives Influence Chain-of-Thought Monitorability?
Matt MacDermott, Qiyao Wei, Rada Djoneva +1
AI systems that output their reasoning in natural language offer an opportunity for safety -- we can \emph{monitor} their chain of thought (CoT) for undesirable reasoning, such as…
Semantic-KG: Using Knowledge Graphs to Construct Benchmarks for Measuring Semantic Similarity
Qiyao Wei, Edward Morrell, Lea Goetz +1
Evaluating the open-form textual responses generated by Large Language Models (LLMs) typically requires measuring the semantic similarity of the response to a (human generated) ref…
Paper Copilot: Tracking the Evolution of Peer Review in AI Conferences
Jing Yang, Qiyao Wei, Jiaxin Pei
The rapid growth of AI conferences is straining an already fragile peer-review system, leading to heavy reviewer workloads, expertise mismatches, inconsistent evaluation standards,…
Event-Aware Sentiment Factors from LLM-Augmented Financial Tweets: A Transparent Framework for Interpretable Quant Trading
Yueyi Wang, Qiyao Wei
In this study, we wish to showcase the unique utility of large language models (LLMs) in financial semantic annotation and alpha signal discovery. Leveraging a corpus of company-re…
Statistical Hypothesis Testing for Auditing Robustness in Language Models
Paulius Rauba, Qiyao Wei, Mihaela van der Schaar
Consider the problem of testing whether the outputs of a large language model (LLM) system change under an arbitrary intervention, such as an input perturbation or changing the mod…
Position: The ML Community Must Build an AI-Augmented Peer-Review Ecosystem
Qiyao Wei, Samuel Holt, Jing Yang +2
Peer review, the bedrock of scientific advancement in machine learning (ML), is strained by a crisis of scale. Exponential growth in manuscript submissions to premier ML venues suc…