Domain adaptation of Russian ModernBERT for long legal documents

I. Litvak, D. Gvozdetsky, F. Lashkin +7

We investigate whether continued pretraining on Russian legislative documents improves a Russian ModernBERT encoder on legal text. The adapted model, RuModernBERT-ruLaw, was traine…