Why does in-context learning fail sometimes? Evaluating in-context learning on open and closed questions

Xiang Li, Haoran Tang, Siyu Chen +3

We measure the performance of in-context learning as a function of task novelty and difficulty for open and closed questions. For that purpose, we created a novel benchmark consist…