Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
Asymmetric Within-Document Predictive Learning for Scientific Document Representation
You Zuo, Éric de la Clergerie, Benoît Sagot
We study predictive pretraining for scientific document representation using the discourse structure of papers. We propose SciJEPA, a citation-free framework that learns through as…
cs.CL2026
A Causal Language Modeling Detour Improves Encoder Continued Pretraining
Rian Touchent, Eric de la Clergerie
When adapting an encoder to a new domain, the standard approach is to continue training with Masked Language Modeling (MLM). We show that temporarily switching to Causal Language M…