Showing cs.CLShow all
3 papers · 1 filter
cs.CL2026
Context-level Language Modeling by Learning Predictive Context Embeddings
Beiya Dai, Yuliang Liu, Daozheng Xue +6
We propose ContextLM, a framework that implicitly learns multi-token prediction by augmenting standard pretraining with an intrinsic next-context prediction objective. ContextLM bu…
cs.CL2026
KaLM: Knowledge-aligned Autoregressive Language Modeling via Dual-view Knowledge Graph Contrastive Learning
Peng Yu, Cheng Deng, Beiya Dai +2
Autoregressive large language models (LLMs) pre-trained by next token prediction are inherently proficient in generative tasks. However, their performance on knowledge-driven tasks…
cs.CL2024
GeoGalactica: A Scientific Large Language Model in Geoscience
Zhouhan Lin, Cheng Deng, Le Zhou +18
Large language models (LLMs) have achieved huge success for their general knowledge and ability to solve a wide spectrum of tasks in natural language processing (NLP). Due to their…