Large Language Models Can Explain Experts Better Than Experts Themselves

Abstract

Capturing tacit knowledge, or the "know-how" embedded in individual experiences and contexts, is a central challenge that limits the preservation and transfer of valuable expertise. Traditional methods rely on experts' ability to articulate their own reasoning, even though much of tacit knowledge is difficult to verbalize. Here, we show that Large Language Models (LLMs) can uncover tacit knowledge from expert behavior and support knowledge transfer from experts to novices. We first identify prompting strategies that improve LLMs' ability to externalize tacit knowledge (i.e., transform tacit into explicit knowledge), and then evaluate the downstream utility of the resulting LLM-externalized tacit knowledge. Our analysis reveals that LLM-externalized tacit knowledge incorporating expert behavior improves decision quality and helps reduce the expert-novice score gap, outperforming knowledge articulated by human experts. We further show that LLMs meaningfully learn and extract knowledge from expert behavior rather than simply mimicking expert responses, and that these findings generalize across different LLMs and information retrieval methods. These findings reaffirm Polanyi's Paradox---that "we can know more than we can tell"---and suggest that LLMs can help overcome human experts' articulation bottleneck that has long challenged tacit knowledge externalization.

arXiv commentMain text (29 pages) with Supplementary Information appended (29 pages). Supersedes v1