20 papers
Feed-Forward Steering in Transformer Residual Dynamics
Timur Mudarisov, Mikhail Burtsev, Radu State
Attention-only dynamical theories model Transformer residual directions as particles aggregating on a sphere. We extend this framework by incorporating the feed-forward network (FF…
Geometry-Guided Layerwise FFN Width Allocation in Transformers
Timur Mudarisov, Mikhail Burtsev, Radu State
Feed-forward networks (FFNs) account for a large fraction of Transformer parameters, yet their hidden width is usually constant across depth. We ask whether this capacity can inste…
QeHDC: Hyperdimensional Computing based on Quantum-enhanced binding and SuperClass Construction
Yangjie Xu, Hui Huang, Li Ning +1
Hyperdimensional Computing (HDC) is a robust computational framework inspired by human cognition characterized by simple and efficient operations within high-dimensional vector spa…
Agent Skill Framework: Perspectives on the Potential of Small to Medium Language Models in Industrial Environments
Yangjie Xu, Lujun Li, Lama Sleem +6
Agent skills are widely supported by major agentic frameworks and perform well with proprietary models, yet their effectiveness for small and medium-sized open source language mode…
The Last Visible Pixel: Probing Fine-Scale Perception in Vision-Language Models
Lujun Li, Lama Sleem, Niccolo Gentile +4
Recent vision-language models (VLMs) excel at multimodal understanding and reasoning, yet their fine-grained visual perception remains underexplored. A natural extension of ``How m…
The Necessity of Setting Temperature in LLM-as-a-Judge
Lujun Li, Lama Sleem, Yangjie Xu +4
Using large language models (LLMs) as judges for evaluating model outputs has emerged as an important paradigm for automated evaluation. However, the choice of decoding temperature…