5 citations · 6 across the 7 of their papers we have counts for
6 papers · 1 filter
MultiSynt/MT: Trillion-Token Multi-Parallel Pre-Training Data Translated Across 36 Languages
Maximilian Idahl, Jörg Tiedemann, Sampo Pyysalo +19
Open web-scale pre-training corpora remain concentrated in English, limiting multilingual LLM development. We introduce MultiSynt/MT, an open synthetic parallel corpus with approxi…
On the Limits of Model Merging for Multilinguality in Pre-Training
Seth Aycock, Fedor Vitiugin, Aleksandr Umnov +2
Endowing models with consistent multilingual performance can be achieved by mixing pre-training data, or post-training approaches such as language-specific model merging. In this w…
FIN-bench-v2: A Unified and Robust Benchmark Suite for Evaluating Finnish Large Language Models
Joona Kytöniemi, Jousia Piha, Akseli Reunamo +3
We introduce FIN-bench-v2, a unified benchmark suite for evaluating large language models in Finnish. FIN-bench-v2 consolidates Finnish versions of widely used benchmarks together…
HPLT 3.0: Very Large-Scale Multilingual Resources for LLMs and MT. Mono- and Bi-lingual Data, Multilingual Evaluation, and Pre-Trained Models
Stephan Oepen, Nikolay Arefev, Mikko Aulamo +29
We present an ongoing initiative to provide open, very large, high-quality, and richly annotated textual datasets for almost 200 languages. At 30 trillion tokens, this is likely th…
Unraveling Code-Mixing Patterns in Migration Discourse: Automated Detection and Analysis of Online Conversations on Reddit
Fedor Vitiugin, Sunok Lee, Henna Paakki +2
The surge in global migration patterns underscores the imperative of integrating migrants seamlessly into host communities, necessitating inclusive and trustworthy public services.…
Construction of Hyper-Relational Knowledge Graphs Using Pre-Trained Large Language Models
Preetha Datta, Fedor Vitiugin, Anastasiia Chizhikova +1
Extracting hyper-relations is crucial for constructing comprehensive knowledge graphs, but there are limited supervised methods available for this task. To address this gap, we int…