22 papers
Can Loan Processors and Underwriters Spot AI-Edited Statements? An Empirical Study
Simiao Ren, Xingyu Shen, Tommy Duong +1
AI-assisted editing can produce loan documents that preserve familiar issuer layouts while changing the applicant's identity. We investigated whether experienced underwriters and l…
How Firm Are AI Models' Ethical Boundaries? A Pay-Stub Editing Test of 15 Models
Simiao Ren, Xingyu Shen, Tommy Duong +2
Can a small change in wording move an AI model from refusing a sensitive request to acting on it? We test 15 models on the same request to increase the income shown on a pay stub.…
Open-Jev Judgments on CallScreenBench: Calibrated One-Pass Scam Screening with a Small Language Model
Simiao Ren, Kidus Zewde, Xingyu Shen +6
Screening a phone call for fraud needs a trustworthy probability after every caller turn, in milliseconds. Jev-style typed decisions promise exactly that: declared options go in, o…
Agents That Edit Documents: Measuring Agentic PDF Forgery Against a Non-Agentic Control
Simiao Ren, Ankit Raj, Tommy Duong +6
AI agents that carry a multi-step computer task through on their own became ordinary tools in the past year, and the same autonomy is available to anyone whose task is harmful. We…
ChatGPT Images 2.5 on Forgery Tasks: Testing Advertised Improvements Against Known Answers
Ankit Raj, Yuxin Zhang, Kidus Zewde +7
OpenAI released ChatGPT Images 2.5 on 8 September 2026, advertising more precise local edits, better consistency across edits, more faithful reference products and sharper detail.…
The Machines Are Calling: Measuring Automated and Synthetic Voices in Unwanted Inbound Calls
Xingyu Shen, Tommy Duong, Muduo Xu +8
In February 2024 the U.S. Federal Communications Commission (FCC) placed AI-generated voices under the Telephone Consumer Protection Act (TCPA). Yet no peer-reviewed measurement sa…
A Corpus of Real Scam- and Spam-Call Conversations from an Active Voice-Agent Honeypot
Ethan Traister, Dennis Tsang Ng, Siyu Zhang +7
Real conversations between fraudsters and their targets are among the most informative artifacts for studying telephone scams, yet also the scarcest: passive honeypots overwhelming…
Anatomy of a Scam Call: What 10,000 real scam and spam calls reveal about how phone scammers operate
Ethan Traister, Ankit Raj, Jiaqi Gan +7
Telephone fraud is pervasive and costly, but its inner workings are rarely observed at scale. We analyze a complete corpus of 10,211 inbound scam and spam calls -- 913 hours of aud…
CallScreenBench: Benchmarking Small Language Models as Phone Secretaries
Jiaqi Gan, Haoyuan Tang, Jamey Z. Liang +6
Language models small enough to run on a handset, quantized to a few bits, are increasingly capable of acting on their user's behalf -- which makes on-device task automation newly…
GPT-Image-2 in the Wild: A Twitter Dataset of Self-Reported AI-Generated Images from the First Week of Deployment
Kidus Zewde, Simiao Ren, Xingyu Shen +6
The release of GPT-image-2 by OpenAI marks a watershed moment in AI-generated imagery: the boundary between photographic reality and synthetic content has never been more difficult…
When the Forger Is the Judge: GPT-Image-2 Cannot Recognize Its Own Faked Documents
Jiaqi Wu, Yuchen Zhou, Dennis Tsang Ng +5
OpenAI's GPT-Image-2 has effectively erased the visual boundary between authentic and AI-edited document images: a single number on a receipt can be replaced in under a second for…
A Synthetic Eye Movement Dataset for Script Reading Detection: Real Trajectory Replay on a 3D Simulator
Kidus Zewde, Yuchen Zhou, Dennis Ng +6
Large vision-language models have achieved remarkable capabilities by training on massive internet-scale data, yet a fundamental asymmetry persists: while LLMs can leverage self-su…
Chinese Language Is Not More Efficient Than English in Vibe Coding: A Preliminary Study on Token Cost and Problem-Solving Rate
Simiao Ren, Xingyu Shen, Yuchen Zhou +3
A claim has been circulating on social media and practitioner forums that Chinese prompts are more token-efficient than English for LLM coding tasks, potentially reducing costs by…
GPT4o-Receipt: A Dataset and Human Study for AI-Generated Document Forensics
Yan Zhang, Simiao Ren, Ankit Raj +6
Can humans detect AI-generated financial documents better than machines? We present GPT4o-Receipt, a benchmark of 1,235 receipt images pairing GPT-4o-generated receipts with authen…
DOCFORGE-BENCH: A Comprehensive 0-shot Benchmark for Document Forgery Detection and Analysis
Zengqi Zhao, Weidi Xia, En Wei +7
We present DOCFORGE-BENCH, the first unified zero-shot benchmark for document forgery detection, evaluating 14 methods across eight datasets spanning text tampering, receipt forger…
AIForge-Doc: A Benchmark for Detecting AI-Forged Tampering in Financial and Form Documents
Jiaqi Wu, Yuchen Zhou, Muduo Xu +6
We present AIForge-Doc, the first dedicated benchmark targeting exclusively diffusion-model-based inpainting in financial and form documents with pixel-level annotation. Existing d…
Can a Teenager Fool an AI? Evaluating Low-Cost Cosmetic Attacks on Age Estimation Systems
Xingyu Shen, Tommy Duong, Xiaodong An +6
Age estimation systems are increasingly deployed as gatekeepers for age-restricted online content, yet their robustness to cosmetic modifications has not been systematically evalua…
Out of the box age estimation through facial imagery: A Comprehensive Benchmark of Vision-Language Models vs. out-of-the-box Traditional Architectures
Simiao Ren, Xingyu Shen, Ankit Raj +8
Facial age estimation plays a critical role in content moderation, age verification, and deepfake detection. However, no prior benchmark has systematically compared modern vision-l…
How well are open sourced AI-generated image detection models out-of-the-box: A comprehensive benchmark study
Simiao Ren, Yuchen Zhou, Xingyu Shen +9
As AI-generated images proliferate across digital platforms, reliable detection methods have become critical for combating misinformation and maintaining content authenticity. Whil…
Can Multi-modal (reasoning) LLMs detect document manipulation?
Zisheng Liang, Kidus Zewde, Rudra Pratap Singh +7
Document fraud poses a significant threat to industries reliant on secure and verifiable documentation, necessitating robust detection mechanisms. This study investigates the effic…
Can Multi-modal (reasoning) LLMs work as deepfake detectors?
Simiao Ren, Yao Yao, Kidus Zewde +8
Deepfake detection remains a critical challenge in the era of advanced generative models, particularly as synthetic media becomes more sophisticated. In this study, we explore the…
Do Deepfake Detectors Work in Reality?
Simiao Ren, Hengwei Xu, Tsang Ng +7
Deepfakes, particularly those involving faceswap-based manipulations, have sparked significant societal concern due to their increasing realism and potential for misuse. Despite ra…