1 citations · 1 across the 3 of their papers we have counts for
11 papers
Pando: Do Interpretability Methods Work When Models Won't Explain Themselves?
Ziqian Zhong, Aashiq Muhamed, Mona T. Diab +2
Mechanistic interpretability is often motivated for alignment auditing, where a model's verbal explanations can be absent, incomplete, or misleading. Yet many evaluations do not co…
DSPA: Dynamic SAE Steering for Data-Efficient Preference Alignment
James Wedgwood, Aashiq Muhamed, Mona T. Diab +1
Preference alignment is usually achieved by weight-updating training on preference data, which adds substantial alignment-stage compute and provides limited mechanistic visibility.…
Beyond Understanding: Evaluating the Pragmatic Gap in LLMs' Cultural Processing of Figurative Language
Mena Attia, Aashiq Muhamed, Mai Alkhamissi +2
We present a comprehensive evaluation of the ability of large language models (LLMs) to process culturally grounded language, specifically to understand and pragmatically use figur…
RefusalBench: Generative Evaluation of Selective Refusal in Grounded Language Models
Aashiq Muhamed, Leonardo F. R. Ribeiro, Markus Dreyer +2
The ability of language models in RAG systems to selectively refuse to answer based on flawed context is critical for safety, yet remains a significant failure point. Our large-sca…
CCRS: A Zero-Shot LLM-as-a-Judge Framework for Comprehensive RAG Evaluation
Aashiq Muhamed
RAG systems enhance LLMs by incorporating external knowledge, which is crucial for domains that demand factual accuracy and up-to-date information. However, evaluating the multifac…
NeurIPS 2023 Competition: Privacy Preserving Federated Learning Document VQA
Marlon Tobaben, Mohamed Ali Souibgui, Rubèn Tito +24
The Privacy Preserving Federated Learning Document VQA (PFL-DocVQA) competition challenged the community to develop provably private and communication-efficient solutions in a fede…