11 papers
SpectralCanary: Embarrassingly Simple Low-Frequency Traces for T2I Dataset-Use Auditing
Sy-Tuyen Ho, Anirudh Satheesh, Soham Chawathe +4
Auditing whether a text-to-image (T2I) model was fine-tuned on a particular image collection is difficult: a model can absorb collection-level visual characteristics without reprod…
When AI Reviews Train AI Reviewers: Scientific-Judgment Collapse and Mitigation
Sy-Tuyen Ho, Minghui Liu, Furong Huang +1
LLMs increasingly assist scientific peer review. This creates a feedback loop when generated reviews enter the training data of future LLMs. While recursive training has been studi…
Trait-space Monitoring for Emergent Misalignment During Supervised Finetuning
Huy Nghiem, Sy-Tuyen Ho, Sarah Wiegreffe +1
Emergent misalignment (EM) occurs when narrow finetuning induces dangerous behavior outside the finetuning task. Detecting this shift through repeated behavioral evaluation is cost…
SoundnessBench: Can Your AI Scientist Really Tell Good Research Ideas from Bad Ones?
Sy-Tuyen Ho, Minghui Liu, Huy Nghiem +6
Autonomous AI research agents aim to speed up scientific discovery by automating the research pipeline from hypothesis generation to peer review. However, existing benchmarks rarel…
Bias in the Tails: How Name-conditioned Evaluative Framing in Resume Summaries Destabilizes LLM-based Hiring
Huy Nghiem, Phuong-Anh Nguyen-Le, Sy-Tuyen Ho +1
Research has documented LLMs' name-based bias in hiring and salary recommendations. In this paper, we instead consider a setting where LLMs generate candidate summaries for downstr…
Agentic Critical Training
Weize Liu, Minghui Liu, Sy-Tuyen Ho +5
Imitation learning (IL) teaches language-model agents to reproduce expert actions but not to distinguish them from plausible mistakes. Self-reflection methods expose models to alte…
Do Vision-Language Models Leak What They Learn? Adaptive Token-Weighted Model Inversion Attacks
Ngoc-Bao Nguyen, Sy-Tuyen Ho, Koh Jun Hao +1
Model inversion (MI) attacks pose significant privacy risks by reconstructing private training data from trained neural networks. While prior studies have primarily examined unimod…
Revisiting Model Inversion Evaluation: From Misleading Standards to Reliable Privacy Assessment
Sy-Tuyen Ho, Koh Jun Hao, Ngoc-Bao Nguyen +2
Model Inversion attacks aim to reconstruct information from private training data by exploiting access to a target model. Nearly all recent MI studies evaluate attack success using…
Vision Transformer Neural Architecture Search for Out-of-Distribution Generalization: Benchmark and Insights
Sy-Tuyen Ho, Tuan Van Vo, Somayeh Ebrahimkhani +1
While ViTs have achieved across machine learning tasks, deploying them in real-world scenarios faces a critical challenge: generalizing under OoD shifts. A crucial research gap exi…
On the Vulnerability of Skip Connections to Model Inversion Attacks
Jun Hao Koh, Sy-Tuyen Ho, Ngoc-Bao Nguyen +1
Skip connections are fundamental architecture designs for modern deep neural networks (DNNs) such as CNNs and ViTs. While they help improve model performance significantly, we iden…
Model Inversion Robustness: Can Transfer Learning Help?
Sy-Tuyen Ho, Koh Jun Hao, Keshigeyan Chandrasegaran +2
Model Inversion (MI) attacks aim to reconstruct private training data by abusing access to machine learning models. Contemporary MI attacks have achieved impressive attack performa…