Tool and Trainer? Experimental Evidence on Human Learning from Predictive AI in Cancer Diagnosis

Abstract

Artificial intelligence (AI) is increasingly deployed to aid professional judgment, yet its implications for the development of human expertise remain unclear. We examine how the availability of AI assistance during training and task execution shapes novices' individual and collective judgment in the context of cancer diagnosis. Across two field experiments at a large teaching hospital, 576 medical novices learned to diagnose lung cancer from CT scans. Study 1 used a 2 * 2 factorial design that randomized AI assistance during training, task execution, both, or neither. Novices who received AI assistance during training achieved higher diagnostic accuracy in both AI-assisted and unaided task execution, with a larger improvement in the former. However, training-phase assistance was associated with more common errors than task-execution-phase assistance, suggesting that while the trainer effect improves individual accuracy, it limits the benefits of aggregating judgments. Study 2 shows that AI assistance during earlier task execution improved subsequent unaided performance and reduced common errors in the unaided phase of task execution, thus revealing a surprisingly beneficial "tool-as-trainer" effect. These findings suggest that, contingent on the right deployment strategy, AI assistance can support individual expertise development and improve collective judgment by reducing common errors across individuals.