22 citations · 51 across the 40 of their papers we have counts for
48 papers
When Tokenizers Fail: Byte-Level Chunking for Zero-Shot Transfer to Low-Resource Languages
Sanjeev Kumar, Atsuki Yamaguchi, Nikolaos Aletras
Subword tokenization hinders low-resource language processing by imposing frequency patterns from dominant languages onto script-sharing variants. Byte-level models bypass this iss…
On the Utility and Factual Reliability of Pruned Mixture-of-Experts Models in the Biomedical Domain
Atsuki Yamaguchi, Szymon Palucha, Léo Bijar +2
Mixture-of-Experts (MoE) models offer inference speedups via selective activation but impose substantial memory requirements because the whole network must remain loaded. Structure…
Where does output diversity collapse in post-training?
Constantinos Karouzos, Xingwei Tan, Nikolaos Aletras
Post-trained language models produce less varied outputs than their base counterparts. This output diversity collapse undermines inference-time scaling methods that rely on varied…
Deconstructing Attention: Investigating Design Principles for Effective Language Modeling
Huiyin Xue, Nafise Sadat Moosavi, Nikolaos Aletras
The success of Transformer language models is widely credited to their dot-product attention mechanism, which interweaves a set of key design principles: mixing information across…
Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?
Samuel Lewis-Lim, Xingwei Tan, Zhixue Zhao +1
Recent work has demonstrated that Chain-of-Thought (CoT) often yields limited gains for soft-reasoning problems such as analytical and commonsense reasoning. CoT can also be unfait…
Progressive Depth Up-scaling via Optimal Transport
Mingzi Cao, Xi Wang, Nikolaos Aletras
Scaling Large Language Models (LLMs) yields performance gains but incurs substantial training costs. Depth up-scaling offers training efficiency by adding new layers to pre-trained…