24 citations · 27 across the 9 of their papers we have counts for
11 papers
Laguna M.1/XS.2 Technical Report
Julien Abadji, Marah Abdin, Connor Adams +93
We present Laguna M.1 and Laguna XS.2, two Mixture-of-Experts foundation models built for long-horizon, agentic coding: M.1 has B total parameters (B activated per tok…
SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Masks
Fenia Christopoulou, Ronald Cardenas, Gerasimos Lampouras +2
Direct alignment algorithms have proven an effective step for aligning language models to human-desired behaviors. Current variants of the Direct Preference Optimization objective…
Human-inspired Episodic Memory for Infinite Context LLMs
Zafeirios Fountas, Martin A Benfeghoul, Adnan Oomerjee +4
Large language models (LLMs) have shown remarkable capabilities, but still struggle with processing extensive contexts, limiting their ability to maintain coherence and accuracy ov…
Text-to-Code Generation with Modality-relative Pre-training
Fenia Christopoulou, Guchun Zhang, Gerasimos Lampouras
Large pre-trained language models have recently been expanded and applied to programming language tasks with great success, often through further pre-training of a strictly-natural…
Training Dynamics for Curriculum Learning: A Study on Monolingual and Cross-lingual NLU
Fenia Christopoulou, Gerasimos Lampouras, Ignacio Iacobacci
Curriculum Learning (CL) is a technique of training models via ranking examples in a typically increasing difficulty trend with the aim of accelerating convergence and improving ge…
EntityCS: Improving Zero-Shot Cross-lingual Transfer with Entity-Centric Code Switching
Chenxi Whitehouse, Fenia Christopoulou, Ignacio Iacobacci
Accurate alignment between languages is fundamental for improving cross-lingual pre-trained language models (XLMs). Motivated by the natural phenomenon of code-switching (CS) in mu…