10 citations · 20 across the 18 of their papers we have counts for
9 papers · 1 filter
Attention Sinks in Diffusion Language Models
Maximo Eduardo Rulli, Simone Petruzzi, Edoardo Michielon +3
Masked Diffusion Language Models (DLMs) have recently emerged as a promising alternative to traditional Autoregressive Models (ARMs). DLMs employ transformer encoders with bidirect…
Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution
Alessio Devoto, Maximilian Jeblick, Simon Jégou
Memory consumption of the Key-Value (KV) cache represents a major bottleneck for efficient large language model inference. While attention-score-based KV cache pruning shows promis…
Universal Properties of Activation Sparsity in Modern Large Language Models
Filip Szatkowski, Patryk Będkowski, Alessio Devoto +5
Activation sparsity is an intriguing property of deep neural networks that has been extensively studied in ReLU-based models, due to its advantages for efficiency, robustness, and…
From KMMLU-Redux to KMMLU-Pro: A Professional Korean Benchmark Suite for LLM Evaluation
Seokhee Hong, Sunkyoung Kim, Guijin Son +3
The development of Large Language Models (LLMs) requires robust benchmarks that encompass not only academic domains but also industrial fields to effectively evaluate their applica…
Production, Quality Assurance and Quality Control of the SiPM Tiles for the DarkSide-20k Time Projection Chamber
F. Acerbi, P. Adhikari, P. Agnes +301
The DarkSide-20k dark matter direct detection experiment will employ a 21 m^2 silicon photomultiplier (SiPM) array, instrumenting a dual-phase 50 tonnes liquid argon Time Projectio…
Interpretable Classification of Levantine Ceramic Thin Sections via Neural Networks
Sara Capriotti, Alessio Devoto, Simone Scardapane +2
Classification of ceramic thin sections is fundamental for understanding ancient pottery production techniques, provenance, and trade networks. Although effective, traditional petr…
Adaptive Semantic Token Communication for Transformer-based Edge Inference
Alessio Devoto, Jary Pomponi, Mattia Merluzzi +2
This paper presents an adaptive framework for edge inference based on a dynamically configurable transformer-powered deep joint source channel coding (DJSCC) architecture. Motivate…
Q-Filters: Leveraging QK Geometry for Efficient KV Cache Compression
Nathan Godey, Alessio Devoto, Yu Zhao +4
Autoregressive language models rely on a Key-Value (KV) Cache, which avoids re-computing past hidden states during generation, making it faster. As model sizes and context lengths…
Mixture-of-Experts Graph Transformers for Interpretable Particle Collision Detection
Donatella Genovese, Alessandro Sgroi, Alessio Devoto +6
The Large Hadron Collider at CERN produces immense volumes of complex data from high-energy particle collisions, demanding sophisticated analytical techniques for effective interpr…