8 papers
Looking in the Mirror: Introspecting Side-Effect Misalignments Induced by Fine-Tuning
Kotaro Yoshida, Laura Gomezjurado Gonzalez, Yukinori Yamamoto +3
Fine-tuning enables a source model to acquire desired capabilities and behaviors in a target domain while retaining much of its general-purpose competence. However, this adaptation…
DisTaC: Conditioning Task Vectors via Distillation for Robust Model Merging
Kotaro Yoshida, Yuji Naraki, Takafumi Horie +2
Model merging has emerged as an efficient and flexible paradigm for multi-task learning, with numerous methods being proposed in recent years. However, these state-of-the-art techn…
On Fairness of Task Arithmetic: The Role of Task Vectors
Hiroki Naganuma, Kotaro Yoshida, Laura Gomezjurado Gonzalez +3
Model editing techniques, particularly task arithmetic with task vectors, offer an efficient alternative to full fine-tuning by enabling direct parameter updates through simple ari…
Generalization Measures under Controlled Covariate Shift: A Regime-Aware Benchmark
Sora Nakai, Youssef Fadhloun, Kacem Mathlouthi +4
Predicting generalization from quantities available before target-test evaluation remains a central challenge in deep learning. The systematic benchmark of Jiang et al. (2020) eval…
How Does Preconditioning Guide Feature Learning in Deep Neural Networks?
Kotaro Yoshida, Atsushi Nitanda
Preconditioning is widely used in machine learning to accelerate convergence on the empirical risk, yet its role on the expected risk remains underexplored. In this work, we invest…
Robust Invariant Representation Learning by Distribution Extrapolation
Kotaro Yoshida, Konstantinos Slavakis
Invariant risk minimization (IRM) aims to enable out-of-distribution (OOD) generalization in deep learning by learning invariant representations. As IRM poses an inherently challen…