3 citations · 5 across the 20 of their papers we have counts for
10 papers · 1 filter
Retrieval Capacity of Self-Attention Under Competition
Timur Mudarisov, Mikhail Burtsev, Radu State
How many tokens from its context does a language model actually use, and what determines that number? We study this question through self-attention. Without retraining, we retain o…
The Geometry of Inference in Transformer Residual Streams
Timur Mudarisov, Mikhail Burtsev, Radu State
Transformer language models build predictions through successive residual updates, but how their representations become specific to an eventual outcome remains unclear. We study th…
Agents Don't Paginate: First-Chunk Selection for LLM Tool Responses
Tatiana Petrova, Andrei Mazniak, Radu State
Coding agents built on large language models (LLMs), such as Claude Code, Cursor, OpenAI Codex, GitHub Copilot, and Aider, receive tool responses that routinely exceed the agent's…
How Much Does Persuasion Strategy Matter? LLM-Annotated Evidence from Charitable Donation Dialogues
Tatiana Petrova, Stanislav Sokol, Radu State
Which persuasion strategies, if any, are associated with donation compliance? Answering this requires fine-grained strategy labels across a full corpus and statistical tests correc…
The Necessity of Setting Temperature in LLM-as-a-Judge
Lujun Li, Lama Sleem, Yangjie Xu +4
Large Language Models (LLMs) are increasingly used as automated judges, yet decoding temperature is often set by intuition rather than evidence. We run over one million large scale…
Do Large Language Models Grasp The Grammar? Evidence from Grammar-Book-Guided Probing in Luxembourgish
Lujun Li, Yewei Song, Lama Sleem +7
Grammar refers to the system of rules that governs the structural organization and the semantic relations among linguistic units such as sentences, phrases, and words within a give…
Small Language Models in the Real World: Insights from Industrial Text Classification
Lujun Li, Lama Sleem, Niccolo' Gentile +2
With the emergence of ChatGPT, Transformer models have significantly advanced text classification and related tasks. Decoder-only models such as Llama exhibit strong performance an…
Exploring the Impact of Temperature on Large Language Models:Hot or Cold?
Lujun Li, Lama Sleem, Niccolo' Gentile +2
The sampling temperature, a critical hyperparameter in large language models (LLMs), modifies the logits before the softmax layer, thereby reshaping the distribution of output toke…
Is Small Language Model the Silver Bullet to Low-Resource Languages Machine Translation?
Yewei Song, Lujun Li, Cedric Lothritz +6
Low-resource languages (LRLs) lack sufficient linguistic resources and are underrepresented in benchmark datasets, resulting in persistently lower translation quality than high-res…
LongKey: Keyphrase Extraction for Long Documents
Jeovane Honorio Alves, Radu State, Cinthia Obladen de Almendra Freitas +1
In an era of information overload, manually annotating the vast and growing corpus of documents and scholarly papers is increasingly impractical. Automated keyphrase extraction add…