20 papers
Evaluation Awareness in Language Models: Representation, Verbalization, and Control
Farzaneh Heidari, Amin Memarian, Guillaume Rabusseau
Both capability and safety benchmarks rest upon the assumption that the behavior of language models undergoing a test is informative about their behavior in deployment. This assump…
Generative Learning for Quantum Measurement Design
Jun Dai, Olivier Nahman-Lévesque, Guillaume Rabusseau +2
Extracting quantum information from a quantum state is a fundamental task of quantum computation, often requiring the estimation of many non-commuting observables under a finite me…
From Expressivity to Sample Complexity: Narrow Teachers for Transformers via C-RASP
Michael Rizvi-Martel, Satwik Bhattamishra, Guillaume Rabusseau +1
A theoretical understanding of Transformers is crucial to better understand the capacities and limitations of large language models (LLMs). There is much work analyzing the express…
TN-SHAP-G: Graph-Structured Tensor Network Surrogates for Shapley Values and Interactions
Farzaneh Heidari, Guillaume Rabusseau
Shapley values are a widely used tool for attributing importance and interactions among input variables in black-box models, but their computation involves a function defined over…
Tensor Cookbook: Mastering Tensors through Diagrams
Beheshteh T. Rakhshan, Guillaume Rabusseau
High-dimensional data arise naturally in many areas of science and engineering, including machine learning, signal processing, computational physics, and statistics. Such data are…
Orth-Dion: Eliminating Geometric Mismatch in Distributed Low-Rank Spectral Optimization
Tatsuhiro Nakamori, Laura Gomezjurado Gonzalez, Ganesh Talluri +5
Low-rank gradient compression reduces communication in distributed training by representing updates with rank- factors. Dion is a recent method that approximates Muon, a spectra…