2 papers
cs.AI2026
OctoLong: Mid-Training On Cross-Repository Code Contexts Enhances Long-Context Modeling
Indraneil Paul, Falko Helm, Goran Glavaš +1
Context lengths of language models (LMs) have dramatically increased, driven by the demands for in-context learning, self-improvement, and long-horizon agentic workflows. Existing…
cs.CL2025
Token Weighting for Long-Range Language Modeling
Falko Helm, Nico Daheim, Iryna Gurevych
Many applications of large language models (LLMs) require long-context understanding, but models continue to struggle with such tasks. We hypothesize that conventional next-token p…