8 papers · 1 filter
MoGround: Measuring and Mitigating Modality Distraction in Vision-Language Models
Luca Zhou, Bo Zhao, Rose Yu +2
We release MoGround, a vision-language dataset spanning four visual domains in which the answer to every question is guaranteed to be available from exactly one modality. This guar…
On Emergent Capabilities and Model Merging
Luca Zhou, Emanuele Rodolà
Fine-tuned checkpoints and adapters now fill public repositories, and the most common operation applied to these artifacts is model merging: arithmetic on their weights that assemb…
The Undetected Damage of Quantization on Retrieval and How to Fix It
Luca Zhou, Alessandro Zirilli, Daniele Solombrino +2
We show that a quantized model that keeps its classification accuracy still changes to of its top-1 retrieval results, and that aggregate ranking metrics reveal only pa…
Hard or Just Unreached? Diagnosing the Sampling Blind Spot in Math-Reasoning Difficulty Estimation
Luca Zhou, Sajel Shah, Emanuele Rodolà +1
Math and science reasoning benchmarks rely on pass@k, the fraction of sampled chains that reach gold, as the canonical per-example difficulty signal. The same signal drives RL with…
Tail-Shape Estimation in LLM Evaluation Is Fragile: A Protocol for Diagnosing False Positives
Luca Zhou
Recent work motivates moving large language model (LLM) evaluation from mean-based to tail-aware metrics, including conditional value-at-risk and tail-index estimates of reward-mod…
Demystifying Mergeability: Interpretable Properties to Predict Model Merging Success
Luca Zhou, Bo Zhao, Rose Yu +1
Model merging combines knowledge from separately fine-tuned models, yet the factors driving its success remain poorly understood. While recent work treats mergeability as an intrin…
On Task Vectors and Gradients
Luca Zhou, Daniele Solombrino, Donato Crisostomi +4
Task arithmetic has emerged as a simple yet powerful technique for model merging, enabling the combination of multiple finetuned models into one. Despite its empirical success, a c…
ATM: Improving Model Merging by Alternating Tuning and Merging
Luca Zhou, Daniele Solombrino, Donato Crisostomi +3
Model merging has emerged as a cost-efficient approximation to multitask learning. Among merging strategies, task arithmetic is notable for its simplicity and effectiveness. In thi…