5 papers
Agents' Last Exam
Yiyou Sun, Xinyang Han, Weichen Zhang +306
Recent AI systems have achieved strong results on a wide range of benchmarks, yet these gains have not translated into economically meaningful deployment across many professional d…
ADR: An Agentic Detection System for Enterprise Agentic AI Security
Chenning Li, Pan Hu, Justin Xu +9
We present the Agentic AI Detection and Response (ADR) system, the first large-scale, production-proven enterprise framework for securing AI agents operating through the Model Cont…
Improving the Performance of Radiology Report De-identification with Large-Scale Training and Benchmarking Against Cloud Vendor Methods
Eva Prakash, Maayane Attias, Pierre Chambon +5
Objective: To enhance automated de-identification of radiology reports by scaling transformer-based models through extensive training datasets and benchmarking performance against…
RadEval: A framework for radiology text evaluation
Justin Xu, Xi Zhang, Javid Abderezaei +9
We introduce RadEval, a unified, open-source framework for evaluating radiology texts. RadEval consolidates a diverse range of metrics, from classic n-gram overlap (BLEU, ROUGE) an…
Scaling laws for activation steering with Llama 2 models and refusal mechanisms
Sheikh Abdur Raheem Ali, Justin Xu, Ivory Yang +3
As large language models (LLMs) evolve in complexity and capability, the efficacy of less widely deployed alignment techniques are uncertain. Building on previous work on activatio…