5 papers
An Exploratory Ablation of a Small MLA--SSM Hybrid Language Model
Christos Koutsiaris
We report an exploratory, single-seed ablation of TALH (Adaptive Latent Hybrid), a decoder-only language model with parallel Multi-head Latent Attention (MLA) and a custom recurren…
Nested Byte-Level Vocabularies Are Cheap to Deploy and Expensive to Share: A Pre-Registered Negative Result
Christos Koutsiaris
A byte-level BPE tokenizer is an ordered list of merge rules, so applying only a prefix yields a vocabulary whose token identifiers are the first rows of the full vocabulary. This…
Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference
Christos Koutsiaris
Small language models are usually built like large ones and then squeezed onto a CPU afterwards. We did the opposite: we fixed the target first, one user, one token at a time, 4-bi…
A Contextual Help Browser Extension to Assist Digital Illiterate Internet Users
Christos Koutsiaris
This paper describes the design, implementation, and evaluation of a browser extension that provides contextual help to users who hover over technological acronyms and abbreviation…
Intent-Driven Dynamic Chunking: Segmenting Documents to Reflect Predicted Information Needs
Christos Koutsiaris
Breaking long documents into smaller segments is a fundamental challenge in information retrieval. Whether for search engines, question-answering systems, or retrieval-augmented ge…