Extracting Algorithms in Pre-trained LLMs: A Case on Hidden Markov Models

Yijia Dai, Zhaolin Gao, Yahya Sattar +2

Large language models (LLMs) display a striking ability to predict next observations from Hidden Markov Models (HMMs) via in-context learning (ICL), but the algorithm underlying th…