2 papers
cs.CL2024
DENIAHL: In-Context Features Influence LLM Needle-In-A-Haystack Abilities
Hui Dai, Dan Pechi, Xinyi Yang +2
The Needle-in-a-haystack (NIAH) test is a general task used to assess language models' (LMs') abilities to recall particular information from long input context. This framework how…
cs.CL2023
Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale
Vijeta Deshpande, Dan Pechi, Shree Thatte +2
In recent years, language models have drastically grown in size, and the abilities of these models have been shown to improve with scale. The majority of recent scaling laws studie…