7 papers
Trait-space Monitoring for Emergent Misalignment During Supervised Finetuning
Huy Nghiem, Sy-Tuyen Ho, Sarah Wiegreffe +1
Emergent misalignment (EM) occurs when narrow finetuning causes a model to behave dangerously outside the finetuning task. Standard training signals can miss this shift, making rel…
SoundnessBench: Can Your AI Scientist Really Tell Good Research Ideas from Bad Ones?
Sy-Tuyen Ho, Minghui Liu, Huy Nghiem +1
Autonomous AI research agents aim to accelerate scientific discovery by automating the research pipeline, from hypothesis generation to peer review. However, existing benchmarks ra…
Revisiting Model Inversion Evaluation: From Misleading Standards to Reliable Privacy Assessment
Sy-Tuyen Ho, Koh Jun Hao, Ngoc-Bao Nguyen +2
Model Inversion attacks aim to reconstruct information from private training data by exploiting access to a target model. Nearly all recent MI studies evaluate attack success using…
Bias in the Tails: How Name-conditioned Evaluative Framing in Resume Summaries Destabilizes LLM-based Hiring
Huy Nghiem, Phuong-Anh Nguyen-Le, Sy-Tuyen Ho +1
Research has documented LLMs' name-based bias in hiring and salary recommendations. In this paper, we instead consider a setting where LLMs generate candidate summaries for downstr…
Agentic Critical Training
Weize Liu, Minghui Liu, Sy-Tuyen Ho +3
Training large language models (LLMs) as autonomous agents often begins with imitation learning, but it only teaches agents what to do without understanding why: agents never contr…
Do Vision-Language Models Leak What They Learn? Adaptive Token-Weighted Model Inversion Attacks
Ngoc-Bao Nguyen, Sy-Tuyen Ho, Koh Jun Hao +1
Model inversion (MI) attacks pose significant privacy risks by reconstructing private training data from trained neural networks. While prior studies have primarily examined unimod…