6 papers · 1 filter
GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment
Minxuan Lv, Tiehua Mei, Tanlong Du +9
We present GoLongRL, a fully open-source, capability-oriented post-training recipe for long-context reinforcement learning with verifiable rewards (RLVR). Existing long-context RL…
SemPA: Improving Sentence Embeddings of Large Language Models through Semantic Preference Alignment
Ziyang Chen, Zhenxuan Huang, Yile Wang +3
Traditional sentence embedding methods employ token-level contrastive learning on non-generative pre-trained models. Recently, there have emerged embedding methods based on generat…
LongBench Pro: A More Realistic and Comprehensive Bilingual Long-Context Evaluation Benchmark
Ziyang Chen, Xing Wu, Junlong Jia +4
The rapid expansion of context length in large language models (LLMs) has outpaced existing evaluation benchmarks. Current long-context benchmarks often trade off scalability and r…
EntropyLong: Effective Long-Context Training via Predictive Uncertainty
Junlong Jia, Ziyang Chen, Xing Wu +5
Training long-context language models to capture long-range dependencies requires specialized data construction. Current approaches, such as generic text concatenation or heuristic…
LiteLong: Resource-Efficient Long-Context Data Synthesis for LLMs
Junlong Jia, Xing Wu, Chaochen Gao +8
High-quality long-context data is essential for training large language models (LLMs) capable of processing extensive documents, yet existing synthesis approaches using relevance-b…
Using Prompts to Guide Large Language Models in Imitating a Real Person's Language Style
Ziyang Chen, Stylios Moscholios
Large language models (LLMs), such as GPT series and Llama series have demonstrated strong capabilities in natural language processing, contextual understanding, and text generatio…