Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
Mergeable Model-Side Aggregation States for Long-Context Language Models
Dachuan Song, Junyu Yin, Zechen Hu +1
A known limitation of long-context language models is their increasingly unreliable performance in non-additive, set-based aggregation as context length grows. Examples include car…
cs.CL2026
Smooth Scaling Laws Hide Stepwise Token Learning
Pingjie Wang, Zechen Hu, Peiru Yang +2
Language model loss follows remarkably regular scaling laws over model and data size, yet it remains unclear why the aggregate loss should exhibit a power-law form. Existing explan…