Position: Let's Develop Data Probes to Fundamentally Understand How Data Affects LLM Performance

Shiqiang Wang, Herbert Woisetschläger, Hans Arno Jacobsen +1

Data is fundamental to large language models (LLMs). However, understanding of what makes certain data useful for different stages of an LLM workflow, including training, tuning, a…