3 citations · 5 across the 20 of their papers we have counts for
25 papers
Retrieval Capacity of Self-Attention Under Competition
Timur Mudarisov, Mikhail Burtsev, Radu State
How many tokens from its context does a language model actually use, and what determines that number? We study this question through self-attention. Without retraining, we retain o…
The Geometry of Inference in Transformer Residual Streams
Timur Mudarisov, Mikhail Burtsev, Radu State
Transformer language models build predictions through successive residual updates, but how their representations become specific to an eventual outcome remains unclear. We study th…
Predictive Geometry of Hidden Trajectories in Transformers
Timur Mudarisov, Mikhail Burtsev, Tatiana Petrova +1
Decoder-only transformers are trained only through a terminal next-token prediction loss, yet this loss constrains every intermediate hidden state through the fixed downstream comp…
HINT-Plan: Human Intention-Aware Robot Task Planning in Context-Rich Environments using Vision Language Models
Yuchen Liu, Luigi Palmieri, Lujun Li +3
Approaches to incorporating human awareness into mobile robot decision-making mainly focus on collision avoidance in low-level motion planning, often overlooking the challenges pos…
Feed-Forward Steering in Transformer Residual Dynamics
Timur Mudarisov, Mikhail Burtsev, Radu State
Attention-only dynamical theories model Transformer residual directions as particles aggregating on a sphere. We extend this framework by incorporating the feed-forward network (FF…
Geometry-Guided Layerwise FFN Width Allocation in Transformers
Timur Mudarisov, Mikhail Burtsev, Radu State
Feed-forward networks (FFNs) account for a large fraction of Transformer parameters, yet their hidden width is usually constant across depth. We ask whether this capacity can inste…
Agents Don't Paginate: First-Chunk Selection for LLM Tool Responses
Tatiana Petrova, Andrei Mazniak, Radu State
Coding agents built on large language models (LLMs), such as Claude Code, Cursor, OpenAI Codex, GitHub Copilot, and Aider, receive tool responses that routinely exceed the agent's…
QeHDC: Hyperdimensional Computing based on Quantum-enhanced binding and SuperClass Construction
Yangjie Xu, Hui Huang, Li Ning +1
Hyperdimensional Computing (HDC) is a robust computational framework inspired by human cognition characterized by simple and efficient operations within high-dimensional vector spa…
The Last Visible Pixel: Probing Fine-Scale Perception in Vision-Language Models
Lujun Li, Lama Sleem, Niccolo Gentile +4
Recent vision-language models (VLMs) excel at multimodal understanding and reasoning, yet their fine-grained visual perception remains underexplored. A natural extension of ``How m…
Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory
Tatiana Petrova, Evgeny Polyachenko, Radu State
We study the thermodynamic memory capacity of modern Hopfield networks (Dense Associative Memory models) with continuous states under geometric constraints, extending classical ana…
Friction-Augmented Drifting Models for Resource-Efficient Domain Translation
Arkadii Kazanskii, Tatiana Petrova, Andrey Ustyuzhanin +3
Single-step generators promise high-fidelity synthesis at a fraction of the inference and training cost of ordinary differential equation (ODE)-based flow models, a central concern…
How Much Does Persuasion Strategy Matter? LLM-Annotated Evidence from Charitable Donation Dialogues
Tatiana Petrova, Stanislav Sokol, Radu State
Which persuasion strategies, if any, are associated with donation compliance? Answering this requires fine-grained strategy labels across a full corpus and statistical tests correc…
The Necessity of Setting Temperature in LLM-as-a-Judge
Lujun Li, Lama Sleem, Yangjie Xu +4
Large Language Models (LLMs) are increasingly used as automated judges, yet decoding temperature is often set by intuition rather than evidence. We run over one million large scale…
Agent Skill Framework: Perspectives on the Potential of Small to Medium Language Models in Industrial Environments
Yangjie Xu, Lujun Li, Lama Sleem +7
Skills are getting increasingly used in recent agentic frameworks, performing robustly with proprietary LLMs. Yet, their effectiveness for small and medium-sized open-source models…
Geometric Analysis of Token Selection in Multi-Head Attention
Timur Mudarisov, Mikhal Burtsev, Tatiana Petrova +1
We present a geometric framework for analysing multi-head attention in large language models (LLMs). Without altering the mechanism, we view standard attention through a top-N sele…
Temporal-Spatial Tubelet Embedding for Cloud-Robust MSI Reconstruction using MSI-SAR Fusion: A Multi-Head Self-Attention Video Vision Transformer Approach
Yiqun Wang, Lujun Li, Meiru Yue +1
Cloud cover in multispectral imagery (MSI) significantly hinders early-season crop mapping by corrupting spectral information. Existing Vision Transformer(ViT)-based time-series re…
NegBLEURT Forest: Leveraging Inconsistencies for Detecting Jailbreak Attacks
Lama Sleem, Jerome Francois, Lujun Li +3
Jailbreak attacks designed to bypass safety mechanisms pose a serious threat by prompting LLMs to generate harmful or inappropriate content, despite alignment with ethical guidelin…
Do Large Language Models Grasp The Grammar? Evidence from Grammar-Book-Guided Probing in Luxembourgish
Lujun Li, Yewei Song, Lama Sleem +7
Grammar refers to the system of rules that governs the structural organization and the semantic relations among linguistic units such as sentences, phrases, and words within a give…
Uncovering Zero-Shot Generalization Gaps in Time-Series Foundation Models Using Real-World Videos
Lujun Li, Lama Sleem, Yiqun Wang +3
Recent research on time-series foundation models (TSFMs) has underscored the scarcity of real-world data, often supplemented with synthetic sources in existing datasets, whose gene…
Limitations of Normalization in Attention Mechanism
Timur Mudarisov, Mikhail Burtsev, Tatiana Petrova +1
This paper investigates the limitations of the normalization in attention mechanisms. We begin with a theoretical framework that enables the identification of the model's selective…
Small Language Models in the Real World: Insights from Industrial Text Classification
Lujun Li, Lama Sleem, Niccolo' Gentile +2
With the emergence of ChatGPT, Transformer models have significantly advanced text classification and related tasks. Decoder-only models such as Llama exhibit strong performance an…
Exploring the Impact of Temperature on Large Language Models:Hot or Cold?
Lujun Li, Lama Sleem, Niccolo' Gentile +2
The sampling temperature, a critical hyperparameter in large language models (LLMs), modifies the logits before the softmax layer, thereby reshaping the distribution of output toke…
Vision Transformer-Based Time-Series Image Reconstruction for Cloud-Filling Applications
Lujun Li, Yiqun Wang, Radu State
Cloud cover in multispectral imagery (MSI) poses significant challenges for early season crop mapping, as it leads to missing or corrupted spectral information. Synthetic aperture…
Is Small Language Model the Silver Bullet to Low-Resource Languages Machine Translation?
Yewei Song, Lujun Li, Cedric Lothritz +6
Low-resource languages (LRLs) lack sufficient linguistic resources and are underrepresented in benchmark datasets, resulting in persistently lower translation quality than high-res…