34 citations · 93 across the 19 of their papers we have counts for
Showing 2024 · cs.CLShow all
3 papers · 2 filters
cs.CL2024
How much do contextualized representations encode long-range context?
Simeng Sun, Cheng-Ping Hsieh
We analyze contextual representations in neural autoregressive language models, emphasizing long-range contexts that span several thousand tokens. Our methodology employs a perturb…
cs.CL2024
Suri: Multi-constraint Instruction Following for Long-form Text Generation
Chau Minh Pham, Simeng Sun, Mohit Iyyer
Existing research on instruction following largely focuses on tasks with simple instructions and short responses. In this work, we explore multi-constraint instruction following fo…
cs.CL2024★ 13 cited
RULER: What's the Real Context Size of Your Long-Context Language Models?
Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman +5
The needle-in-a-haystack (NIAH) test, which examines the ability to retrieve a piece of information (the "needle") from long distractor texts (the "haystack"), has been widely adop…