10 citations · 20 across the 18 of their papers we have counts for
21 papers
NVIDIA-labs OO Agents: Native Python Object-Oriented Agents
Paul Furgale, Severin Klingler, James Nolan +12
Traditional agent development is split across prompt templates, tool schemas, callback code, and workflow graphs. We present NVIDIA Object-Oriented Agents (NOOA), a model-agnostic…
Attention Sinks in Diffusion Language Models
Maximo Eduardo Rulli, Simone Petruzzi, Edoardo Michielon +3
Masked Diffusion Language Models (DLMs) have recently emerged as a promising alternative to traditional Autoregressive Models (ARMs). DLMs employ transformer encoders with bidirect…
Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution
Alessio Devoto, Maximilian Jeblick, Simon Jégou
Memory consumption of the Key-Value (KV) cache represents a major bottleneck for efficient large language model inference. While attention-score-based KV cache pruning shows promis…
Universal Properties of Activation Sparsity in Modern Large Language Models
Filip Szatkowski, Patryk Będkowski, Alessio Devoto +5
Activation sparsity is an intriguing property of deep neural networks that has been extensively studied in ReLU-based models, due to its advantages for efficiency, robustness, and…
From KMMLU-Redux to KMMLU-Pro: A Professional Korean Benchmark Suite for LLM Evaluation
Seokhee Hong, Sunkyoung Kim, Guijin Son +3
The development of Large Language Models (LLMs) requires robust benchmarks that encompass not only academic domains but also industrial fields to effectively evaluate their applica…
Production, Quality Assurance and Quality Control of the SiPM Tiles for the DarkSide-20k Time Projection Chamber
F. Acerbi, P. Adhikari, P. Agnes +301
The DarkSide-20k dark matter direct detection experiment will employ a 21 m^2 silicon photomultiplier (SiPM) array, instrumenting a dual-phase 50 tonnes liquid argon Time Projectio…
Interpretable Classification of Levantine Ceramic Thin Sections via Neural Networks
Sara Capriotti, Alessio Devoto, Simone Scardapane +2
Classification of ceramic thin sections is fundamental for understanding ancient pottery production techniques, provenance, and trade networks. Although effective, traditional petr…
Adaptive Semantic Token Communication for Transformer-based Edge Inference
Alessio Devoto, Jary Pomponi, Mattia Merluzzi +2
This paper presents an adaptive framework for edge inference based on a dynamically configurable transformer-powered deep joint source channel coding (DJSCC) architecture. Motivate…
Q-Filters: Leveraging QK Geometry for Efficient KV Cache Compression
Nathan Godey, Alessio Devoto, Yu Zhao +4
Autoregressive language models rely on a Key-Value (KV) Cache, which avoids re-computing past hidden states during generation, making it faster. As model sizes and context lengths…
Mixture-of-Experts Graph Transformers for Interpretable Particle Collision Detection
Donatella Genovese, Alessandro Sgroi, Alessio Devoto +6
The Large Hadron Collider at CERN produces immense volumes of complex data from high-energy particle collisions, demanding sophisticated analytical techniques for effective interpr…
Goal-oriented Communications based on Recursive Early Exit Neural Networks
Jary Pomponi, Mattia Merluzzi, Alessio Devoto +3
This paper presents a novel framework for goal-oriented semantic communications leveraging recursive early exit models. The proposed approach is built on two key components. First,…
Analysing the Residual Stream of Language Models Under Knowledge Conflicts
Yu Zhao, Xiaotang Du, Giwon Hong +6
Large language models (LLMs) can store a significant amount of factual knowledge in their parameters. However, their parametric knowledge may conflict with the information provided…
Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineering
Yu Zhao, Alessio Devoto, Giwon Hong +6
Large language models (LLMs) can store a significant amount of factual knowledge in their parameters. However, their parametric knowledge may conflict with the information provided…
Adaptive Layer Selection for Efficient Vision Transformer Fine-Tuning
Alessio Devoto, Federico Alvetreti, Jary Pomponi +3
Recently, foundation models based on Vision Transformers (ViTs) have become widely available. However, their fine-tuning process is highly resource-intensive, and it hinders their…
Enhancing High-Energy Particle Physics Collision Analysis through Graph Data Attribution Techniques
A. Verdone, A. Devoto, C. Sebastiani +5
The experiments at the Large Hadron Collider at CERN generate vast amounts of complex data from high-energy particle collisions. This data presents significant challenges due to it…
A Simple and Effective Norm-Based Strategy for KV Cache Compression
Alessio Devoto, Yu Zhao, Simone Scardapane +1
The deployment of large language models (LLMs) is often hindered by the extensive memory requirements of the Key-Value (KV) cache, especially as context lengths increase. Existing…
Are We Done with MMLU?
Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong +13
Maybe not. We identify and analyse errors in the popular Massive Multitask Language Understanding (MMLU) benchmark. Even though MMLU is widely adopted, our analysis demonstrates nu…
Adaptive Semantic Token Selection for AI-native Goal-oriented Communications
Alessio Devoto, Simone Petruzzi, Jary Pomponi +2
In this paper, we propose a novel design for AI-native goal-oriented communications, exploiting transformer neural networks under dynamic inference constraints on bandwidth and com…
Conditional computation in neural networks: principles and research trends
Simone Scardapane, Alessandro Baiocchi, Alessio Devoto +3
This article summarizes principles and ideas from the emerging area of applying \textit{conditional computation} methods to the design of neural networks. In particular, we focus o…
Class incremental learning with probability dampening and cascaded gated classifier
Jary Pomponi, Alessio Devoto, Simone Scardapane
Humans are capable of acquiring new knowledge and transferring learned knowledge into different domains, incurring a small forgetting. The same ability, called Continual Learning,…
Adaptive Computation Modules: Granular Conditional Computation For Efficient Inference
Bartosz Wójcik, Alessio Devoto, Karol Pustelnik +2
While transformer models have been highly successful, they are computationally inefficient. We observe that for each layer, the full width of the layer may be needed only for a sma…