5 papers
Quantization-Aware Healing: A Practical Recipe for Recovering Compressed, 4-Bit LLMs
Bakbergen Ryskulov, Iker García-Ferrero, David Montero +5
Serving large language models cheaply increasingly means shipping models that are both structurally compressed to a fraction of their parameters and quantized to 4 bits. Together t…
Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss
Bakbergen Ryskulov, Iker García-Ferrero, Iker GarcÃa-Ferrero +7
Small language models are often the only option for deployment under tight latency, cost, and on-premises constraints, but they are rarely trained from scratch: a compressed model…
ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents
Ander Alvarez, Santhiya Rajan, Samuel Mugel +1
Tool-using LLM agents increasingly use the Model Context Protocol (MCP) to answer from heterogeneous evidence sources, including search, APIs, databases, clinical records, and form…
Scaling Laws for Energy Efficiency of Local LLMs
Ander Alvarez, Alessandro Genuardi, Nilotpal Sinha +6
Deploying local large language models and vision-language models on edge devices requires balancing accuracy with constrained computational and energy budgets. Although graphics pr…
Globally optimized SVD compression of LLMs via Fermi-function-based rank selection and gauge fixing
Roman Rausch, David Jansen, Sukhbinder Singh +1
Large Language Models (LLMs) are very demanding in terms of their computational resources. Low-rank decompositions of LLM weights, e.g. via Singular Value Decomposition (SVD), is a…