Concept-aware Data Construction Improves In-context Learning of Language Models

Michal Štefánik, Marek Kadlčík, Petr Sojka

Many recent language models (LMs) are capable of in-context learning (ICL), manifested in the LMs' ability to perform a new task solely from natural-language instruction. Previous…